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Preface 


An external programme evaluation of Cordaid’s, ICCO’s and Plan Netherlands’ Health- 
related programmes for the period 2002-2004, was carried out between September and 
December 2004 by the HLSP (former IHSD) institute based in London. 

All three Co-Financing Agencies (CFAs) commissioned the study and ICCO was the 
leading agency. This synthesis report is based on three country reports about the CFAs 
support to partner organisations in DRC Congo, Nepal and Zambia. 

The aim of the evaluation was to gain insight into the contribution of the CFAs and 
their partner organisations to the improvement of sustainable use of, access to and 
control over health related services for the poorer sections of the population, with a 
special emphasis on women and youth. 


This study is the third in a series of MBN programme evaluations 2003-2006. An 
External Evaluation Reference Group (ERG) assesses the quality of process and results 
of these evaluation studies of the MBN. In annex 1 their assessment of this study can be 
found. p 4 

According to the three CFAs the study contributes significantly to their policies and 
practices in the field of health, gender and rights based approaches. We value the way 
the researchers have demonstrated the richness and diversity of experiences of partner 
organisations and field offices supported in the different contexts, as well as pointing 
out the weaker points of our policies and practice. This study provides new insights 

as well as confirmations of our own analysis and understandings. The evaluation has 
prompted all CFAs to sharpen or revise health-related policies. 


Cordaid, ICCO and Plan Netherlands give follow up on the key issues emerging from this 
study in several ways. In the first place there will be a joint learning session with CFA 
staff and other stakeholders on the finance of health systems. Furthermore tailor-made 
follow up agendas specific for each of our organisations are designed. Themes on the 
agenda’s, amongst others, will be the dialogue with our partners, the ways to improve 
the use of baseline data and information and the participation of beneficiaries. 


Cordaid will continue its support to organisational development capacities, will 
emphasize the improvement of the access to health services and the strenghtening ot 
the demand side, and will continue the strenthening of church umbrella organisations in 
health, especially on the issue of advocacy. 

On the basis of the evaluation ICCO has completed its health policy. The aim of this 
policy is to improve access of vulnerable people to better quality basic health care 
services in general, and more specific to sexual and reproductive health care services. 

- ICCO’s specific challenge is to improve the clarity of health-related policies and 
strategies together with our partners. 

Plan Netherlands seeks to promote an integrated approach towards reducing child, 
adolescent and maternal morbidity and mortality, to promote the combination of home/ 
community based and clinicybased interventions, to develop and implement strategies 
to reach the most marginalized children, adolescents and women. 
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1.1 


Executive Summary 


Background and context 


The HLSP Institute! was commissioned by three of the Dutch Co-Financing Agencies 
(CFAs) - Cordaid, ICCO and Plan Netherlands —- to undertake an evaluation of their 
health sector programme in 2004 as part of their joint institutional learning programme 
and to improve their accountability. The evaluation used a case study methodology, 
making an in-depth assessment of these CFA health-related programmes in three 
countries: Democratic Republic of Congo (DRC), Nepal and Zambia. The evaluation 
team comprised two international consultants, who coordinated all components of 

the evaluation exercise, and three national research teams. The evaluation questions, 
developed by the CFAs, were: 


1 Which visions, strategies and activities do partner organisations have to improve 
access to, use of and control over basic health services by potential users? 


2 What are the qualitative and quantitative results of these programmes and for whom 
do they improve sustainable access to, use of and control over basic health (related) 
services? 


3 To what extent do (complementary) partnerships, participation in networks and/or 
alliances contribute to better performance of the partner organisations and the health 
system in general? 


4 With what results do (complementary) partnerships, participation in networks and/ 
or alliances contribute to a better performance of the partner organisations and the 
health system in general? 


5 What is the added-value, in terms of results and partnership aspects, of the CFAs’ 
support for the partner organisations and their programmes? 


The three countries have poor socio-economic indicators and weak health systems that 
struggle to provide adequate services for their populations. All feature at the low end 
of the UNDP’s Human Development Index, with the majority of the population in each 
country living below the World Bank’s absolute poverty line of US$1 per day. Large 
parts of the population in all three countries are highly dependent on external aid in 
order to maintain a basic standard of living. 


All three countries face major development challenges. Although enjoying political 
stability, Zambia is facing continuing food security difficulties, exacerbated by its severe 
HIV/AIDS epidemic. Two of the countries (DRC and Nepal) are experiencing on-going 
conflict, which has had a critical impact on people’s livelihoods and personal security, 


_ as well as severely disrupting public service delivery. The dynamics of conflict have 


i Known as the Institute for Health Sector Development, IHSD, until April 2005. 
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had the dual effect of making people both more vulnerable and also less easy to reach 


through development assistance. 


Fach of the three CFA organisations evaluated in this report has responded to the 
challenges presented by the national contexts in a variety of ways. The results and 
analysis in this report are presented with these contextual issues in mind. 


In terms of their overall strategies, Plan Netherlands works through Plan field offices 

to fund programmes providing community health and development activities, mainly 
through its Early Childhood Care and Development Centres in Nepal and through 
district-based programmes in Zambia, often working closely with government providers. 
ICCO and Cordaid work with a wide range of both international and national non- 
government partners, both faith-based and secular. The role of non-state providers 

in each of the countries studied is of vital importance to the health of communities. 

The support of the three CFAs to many of these providers represents a significant 
contribution to the health sector (see summary table of partners). 


Summary table: CFA partner participants in the evaluation 


DRC Nepal Zambia 

Cordaid ICCO Cordaid 

Bureau Diocesan d'Oeuvre Medicale United Ministries of Nepal (UMN) Catholic Centre for Justice, Development 
(BDOM) Kinshasa, Kananga and e Yala Urban Health Project (YUPH) and Peace (CCJDP) 

Bukavu e¢ Centre for Community Devel- Christian Health Association of Zambia 
Ecole de Sante Publique (ESP) opment (CCDN) - Makwanpur (CHAZ) 


Comite National Feme et Development 
(CONAFED) 

SPS (Cordaid assistance discontinued) 
Centre Olame 


DRC 


ICCO 
Bureau Oecumenique d'Appui au 
Developpement (BOAD) 


e Sakriya Sewa Samaj (SSS) 

British Nepal Medical Trust (BNMT) 
Centre for Victims of Torture (CVICT) 
WOREC (Women's Rehabilitation 
Centre) 

Community Based Rehabilitation 
Services (CBRS) 


Nepal 


Plan 
Early Childhood Care and Development 
Centres (ECCDs) 


Civil Society for Poverty Reduction (CSPR) 
Diocese of Ndola 
Diocese of Mpika 
Diocese of Mansa 


Zambia 


Plan 
Evaluated district health and 
development projects in 


Centre Medicale Ecumenique (CME) _ Evaluated projects in e Mazubika 
Beni a Makwanpur e Chibombo 
ELIMU ¢ Kathmandu 

Institut Panafricain de Sante 

Communitaire (IPASC ) 

1.2 


Findings, conclusions and recommendations 


Tel Health-related visions, strategies and activities 

Respondents in all three countries, including the government, stakeholders or 
beneficiaries, were highly appreciative of the health-related work of the CFA partner 
organisations or Plan programmes in their areas. Their absence would leave a gaping 


hole in health-related services, especially where partner organisations work in remote 
areas of the country. 


ee bate: strategies and activities of all CFA partners evaluated favour access to, use 
and control over health services by the poor, as set out in their mission statements. 
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Broadly, strategies adopted by CFA partners fall into two categories: those engaged in 
service delivery and training; and those undertaking policy advocacy and influencing 
activities through networking and coalition development, research activities, capacity 
building, information exchange and lobbying of policy makers. However, all CFA 
partners are involved to a greater or lesser extent with advocacy activities, in terms of 
influencing policy and implementation agendas for pro-poor health and development. 
All the CFAs emphasise local capacity building as part of their development strategy, 
whether their partners are working within a complex humanitarian crisis or a more 
stable development situation. 


Key strategies are summarised below. 


Service provision: The CFA contribution by Cordaid and ICCO supports service provision 
by the very influential non-state sector (mostly faith-based organisations or local NGOs 
such as the Yala Urban Health Project in Nepal). Plan is engaged in direct operational 
projects, often in support of local government staff and structures (such as Plan’s 
district-based activities in Zambia and Nepal, through the ECCD programme). 


In the DRC, the faith-based organisations supported by ICCO and Cordaid provide 
essential health services in about half of the country, and have been able to maintain 
their operations remarkably effectively in the face of the considerable challenges 
imposed by conflict. In Nepal, the BNMT is the major provider of health services in 
the eastern part of the country. In Zambia, Cordaid supports partners in the faith- 
based sector, which overall is responsible for about 40% of health services, and also 
contributes to CHAZ, the umbrella organisation for all faith-based providers. 


CFA partners are reaching out to highly marginalised groups whose access to services 
would otherwise be very limited. Targeted beneficiaries of the more specialised service 
delivery NGOs include survivors of violence and torture (eg CVICT in Nepal), women 
(eg WOREC in Nepal and Centre Olame in DRC), disabled people (e.g. CBRS in Nepal), 
people living with HIV/AIDS (Ndola Diocese, Zambia) and the very poor in low income 
urban areas (YUHP in Nepal). In general, however, the impact of these NGOs tends to 
be localised and as yet there has been limited success in building support for scaling up 
proven new approaches. 


YUHP is a notable exception to this. It is providing basic primary health care services 
to the urban poor, factory workers, migrants and internally displaced persons, mainly 
women and children, in Kathmandu’s municipal area. The project has supported the 
building of the municipality's institutional capacity to take over its functions, and has 
created participatory structures and community-based health programmes that are 
effectively transferring to the authorities. In so doing, YUHP has effectively contributed 
to developing new policy and strategy for urban health care in Nepal. 


Coordination and collaboration with government: The majority of service delivery 
partner organisations are working in coordination with government agencies, where 
they complement and strengthen government services such as Plan’s ECCD centres, 
BNMT, and YUHP in urban primary health care in Nepal. Alternatively, they provide 

- services under contract to, or in partnership with the government in areas where the 
public sector is not established, for example through faith-based sectors in Zambia and 
DRC. The training programmes supported through organisations such as ESP and IPASC 
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in DRC, and through the dioceses in Zambia, were also seen to fit well with national 


priorities and programmes. 


These strategies are seeing results in terms of influencing or contributing to government 
policy development (e.g. the work of Centre Olame in DRC, viens and Plan in Nepal and 
Ndola Diocese’s HIV/AIDS work in Zambia). In Nepal, Plan's approach to community 
health and development through the ECCD programme in collaboration with govern- 
ment has resulted in a nationally owned plan for scaling up. ECCDs have been included 
in the national poverty reduction strategy, although Plan could be advocating more 
strongly to ensure that ECCD standards are maintained in rollout plans, such as 
maintaining the nutrition components that benefit children of low-income households. 


Networking and advocacy: CFA support also benefits several smaller but potentially 
influential NGOs, and networking or umbrella organisations, which are engaged 

in policy advocacy on key priorities such as pro-poor policy implementation and 
monitoring (e.g. CSPR and CCJDP, Zambia) and gender equity (CONAFED in DRC, and 
WOREC in Nepal). These organisations are having some success. For example, CCJDP 
and CSPR in Zambia have worked effectively with community-based organisations 
that form parts of their network, while also providing much needed capacity building 
of community groups. Kinshasa-based CONAFED in the DRC is a network of women’s 
organisations active in all provinces. CONAFED is reported to have contributed to the 
increase in number of women parliamentarians, the participation of women in the 
political negotiations as part of the intra-Congolese dialogue at Sun City in South 
Africa, and raising gender as an issue for NGO networks. 


However, the evaluators consistently found that capacity limitations, including in 
strategic planning and policy-advocacy, affected these organisations’ ability to deliver 
more far-reaching results. Partnerships are not always encouraging the scaling up of 
successful activities of their members, nor acting as vehicles for spreading innovative 
practices. In the DRC, different BDOMs appear to have highly divergent strategies for 
addressing similar problems and are not engaging in significant cross-organisational 
learning. CHAZ, while effectively representing member interests in Zambia, is not 
contributing greatly to sharing lessons learnt or innovations. Innovative practices in 
some Plan project communities are not being transmitted to other Plan areas in Nepal. 


Several of the service delivery NGOs are also engaged in partnerships and coalitions 
for advocacy purposes. Centre Olame is part of local partnerships such as the ‘Coalition 
to fight against rape and violence against women’ and the ‘Network for action on 
human rights in Kivu’ (RADHOSKI). Actions in defending girls and women’s rights are 
commended by local authorities. However, specific policies targeting the protection of 
women from violence are lacking in DRC. Wider use of lessons learned from Centre 
Olame’s success in working with violated and traumatised women could provide 


aba evidence for advocating and implementing such policies in a context affected 
y conflict. 


Community participation in development: This is emphasised by partners in their 
Strategies, either by facilitating community management of project activities (e.g. 

the Early Childhood Care and Development Centres for Plan) or through engaging 
community members in various actions (e.g. health promotion and home-based care 

by community groups in DRC or Zambia). In the countries in conflict, DRC and Nepal, 
programmes supported by the CFAs lend a sense of normalcy to very abnormal lives, by 
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bringing communities together to work for their own improvement, while also providing 
much-needed services. In Nepal, active community engagement was found to be an 
important element in keeping programmes running, as the insurgents do not interfere 
with organisations seen as responding to the expressed needs of communities. There are 
significant local differences: in Zambia, as with DRC, community involvement in health 
management committees was highly variable between the dioceses. This in turn affects 
the degree to which community members engage in other actions for improving their 
health and health services. 


A promising example of effective participation in DRC is the performance-based 
contracting (l’approche contractuelle), which is applied by BDOM Bukavu and recently 
introduced in Kananga. Apart from rendering services more effective, an important 
added value of this approach is that it offers great potential to increase community 
participation in managing health services. 


However, it is also the case that few of the service providers have explicit objectives 
concerning beneficiary control over basic health-related services, and tend to see this 

as being mediated more through existing national policies that promote some form of 
community health management committees. Furthermore, there is currently no means 
of monitoring the involvement of poorer sections of the community, although Plan has 
specific objectives for strengthening the voice of children and young people. Anecdotal 
evidence indicates that the people who are better off in most communities are those who 
participate in community committees. 


Governance arrangements in partnership and umbrella groups do not always support 
full participation. For example, WOREC in Nepal is active as an advocacy and capacity- 
building organisation and in providing services through local bodies. However, there 

is little indication from reports and interviews that the local community organisations 
have much input into the strategic direction of WOREC. 


There are also some wider organisational issues that limit the effectiveness of the CFA 
partner strategies described above. 


The evaluation team found that, overall, the CFA partner organisations rarely have 
strategic plans that set out SMART objectives and indicators to help them monitor 
whether their visions, overall strategies and activities are in fact achieving their purpose. 
Few of the service delivery organisations have explicit objectives to improve or monitor 
beneficiary access to or control over basic health-related services, particularly by the 
poorer groups. 


Although some have set service delivery targets, the overall strategic objectives and 
implications for required capacity to deliver them are not always clear, and reporting 
tends to be input and output focused. In the main, it is difficult to demonstrate that 
their organisational visions are leading to outcomes and impact. There are two main 
exceptions. Plan’s programmes, such as the ECCDs, have measurable indicators used to 
assess programme progress against objectives. BNMT, a long-established INGO, has a 
strategic plan for its service delivery objectives, with baseline data established in 2003. 


For those NGOs, umbrella organisations and networks undertaking advocacy activities, 
an additional challenge lies in the need to develop measurable and achievable capacity- 
building and advocacy strategies. Planning and monitoring these kinds of strategies is 


16 EVALUATION OF HEALTH THEME SEPTEMBER-DECEMBER 2004 


t - this is not just an issue affecting CFAs and their partners, but is 


ognised as difficul 
ei hern and southern NGOs. 


a challenging one for many nort 


4 baseline (e.g. mapping existing policy barriers and 
and assessing stakeholder views and analysis). An 
advocacy strategy needs to set out achievable objectives, using proxy and intermediate 
indicators that measure progress along the impact chain. These might include informed 
media coverage that uses the organisation's research, gaining other NGOs’ support, 
developing alliances, recognition by policy makers and elected officials - these are all 
steps along the path to introducing, implementing and monitoring new policy. 


Specific issues include the need for 
targets, possible allies, adversaries; 


For example, it was noted by the evaluation team that in order for CONAFED to make 
a real change in the position of women, the organisation must identify its strategic 
priorities, build strategic alliances with donors and powerful national stakeholders (for 
example the BDOMs and NGO coordination body, CNONG), and build up managerial 
capacity inside their own organisation and networks. 


The need for partner strategies is also linked to a broader strategic issue — that of each 
CFA’s overarching global and national goals. Of the three CFAs, only Plan Netherlands 
has clear country strategies, linked to Plan’s global policies. Neither ICCO nor Cordaid 
have as transparent or explicit a link between their global policies on health and 

the partners and programmes they support in countries. Consequently there is some 
disconnection between what Cordaid and ICCO aspire to support and what health-related 
activities are actually being implemented in countries through their partners. 


This does not imply that ICCO or Cordaid should become operational organisations. 
However, the development of country strategic frameworks, through a process that 
involves dialogue with the partners themselves, would help to guide partner relations 
in future and create more explicit connections between ICCO and Cordaid policy and 
activities implemented by partners. 


Given the challenging country contexts, it must be recognised that the service delivery 
organisations tend to focus on the key priorities of meeting people’s practical rather 
than strategic needs, whether living in the conflict-affected areas of Eastern and 
Central DRC, and of Nepal, or working with communities in Zambia experiencing the 
triple threat of poverty, food insecurity and HIV/AIDS. However, this also creates the 
challenge for the CFAs of how to best influence and work with their partners and field 


staff to develop and implement newer CFA policy on gender equity and rights-based 
approaches. 


Understanding and ownership of rights-based approaches amongst service delivery 
partners tend to be poorly developed. As all the CFAs are increasingly advocating 

for shifting to a rights-based approach in the work they fund, the question of how 
beneficiaries can influence the partner organisations themselves needs to be addressed 


more explicitly, as well as the problems of engaging more socially marginalised groups 
within communities. 


Plan is the most effective at bridging this policy-implementation gap, perhaps not 

Surprisingly, as it can facilitate greater understanding through its staff structure down 
to community level. However, Plan could also do more to strengthen staff capacity to 
challenge societal norms where those norms lead to the continued marginalisation of 
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certain groups. For example, the evaluation found that young people remain excluded 
from local development committees (Zambia) and dalit (‘low caste’) women are 
prohibited from being involved in certain activities (Nepal). 


Cordaid and ICCO’s ways of working give greater ownership to local/national 
organisations, and therefore require more investment in communicating and negotiating 
with partners to build commitment to gender and rights-based policies. Negotiating 
strategies for improving gender equity and challenging norms that disadvantage women 
is especially needed in some Catholic institutions supported by Cordaid, where some 
inconsistency was found in both DRC and Zambia. Women’s participation in health 
committees and management structures remains poor in some dioceses, and many of 
their sexual and reproductive health needs were not or could not be met. There are, 
however, positive examples to draw on, such as the Family Action Programme in BDOM 
Bukavu, which concretely addresses gender concerns. It provides support to couples on 
issues such as sexual and reproductive health, responsible parenting and prevention 

of violence against women, and it strengthens women’s decision-making power. This 
indeed adds value to the services provided. 


More thought also needs to be given to how best to encourage ‘rights-based approaches’ 
in conflict/post-conflict situations. This evaluation found that in the highly sensitised 
contexts of Nepal and Eastern DRC, more nuanced approaches are required. The DRC 
and Nepal governments are not able to fulfil their obligations to citizens claiming 

their ‘right to health’ In the case of Nepal local government staff themselves feel that 
their involvement in service delivery could compromise effective provision of much- 
needed health services. Rather than approaching rights-based approaches from a ‘duty 
versus claim’ dichotomy, partner organisations could be looking at other demand-side 
approaches that help build community confidence to claim their right to health from a 
wide variety of duty holders. 


Recommendations for improving health-related visions, strategies and activities 


All CFAs 

© More concerted support needs to be given to partners (government and non- 
governmental) in organisational development capacities such as strategic planning 
to help them operationalise their goals, and assess their achievements. Expertise for 
doing this exists in all three countries studied in this evaluation, and specific funding 
could be given by CFAs to buy in this support. 

e In-country workshops should be held to work with partners (ICCO and Cordaid) 
and staff (Plan) on how to understand and adapt CFA core policies on gender and 
rights-based approaches (RBA). RBA in particular needs specific attention in Nepal 
and DRC, and the CFAs should look at how to strengthen their partners to build 
community confidence and demand-side strategies. These could include for example, 
strengthening community management of partner health facilities and exploring the 
value of voucher-for-service schemes that target poor and marginalised groups. 


Cordaid and ICCO 

e National frameworks should set out each CFA’s priorities and strategic directions 
in terms of pro-poor development and target groups, and draw out coherent links 
between its policy and the expected results of its support to existing partners, as 
negotiated with them. This latter point, of defining the CFA’s niche with the partners 
is particularly important, given the fact that the CFAs are only one among several 
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donors for some partners. It would also indicate the kinds of inputs wigrinbt <9 
capacity building, support to advocacy etc.) that partners could expect from , . 
Such a framework would create a clear thread that draws an explicit line from globa 
policies and strategies to the local adaptation and implementation of those policies as 
ith partners. 
e ne ie organisational development programme, ICCO and Cordaid should 
continue to support internal organisational and partnership assessment exercises 


such as the one employed during this evaluation, to help their partners evaluate their 


own development needs. 


Cordaid 
e Given the importance of Cordaid’s global gender policy, the CFA needs to negotiate 


more robustly with its Catholic partners to ensure a much more meaningful 
involvement of women within diocesan structures. It should build on models of 
strong female leadership in the Catholic church in other countries and explore how 
these can be adapted with its partners who work to more male-dominated models. 
As mentioned above there are examples of good practice in the case study countries 
to draw upon. However, the evaluation also found that Cordaid in particular needs 
to consider how it collaborates with Catholic-run health service providers that are 
not open to focusing on women’s participation in decision making and that will not 
engage in effective delivery of sexual and reproductive health services in countries 
with high maternal mortality and escalating rates of sexually transmitted infections 
and HIV. Where church partners prove particularly resistant over time, Cordaid 
should as a last resort consider reducing or withdrawing support altogether. 


Plan 

e As part of developing rights-based approaches, Plan should develop a programme of 
support to its own staff, many of whom come from their programme communities, 
to build understanding of the dynamics of social exclusion and ways in which to 
challenge social norms. 


1.2.2. Quantitative and qualitative impact of health-related services 

The evaluation coordinators have not been able to reach a definitive conclusion about 
the health impact in quantitative terms of CFA partners. In all three countries CFA 
partners were on track with meeting their annual activity targets, where they existed. 
However, quantitative impact is particularly difficult to assess amongst Cordaid and 
ICCO partners, due to a lack of quantitative indicators, poor population data, no baseline 
information and poor monitoring data. Plan has a more developed monitoring and 
evaluation system, which has allowed the evaluators to review progress over time. 


All partners and projects supported by the three CFAs are providing health-related 
services that otherwise would not exist at all, or to a very limited extent in all three 
countries. The community outreach programmes of the CFA partners are particularly 
effective in reaching people with primary health care and health promotion activities, 
where they would otherwise not be reached. For example, the Zambian AIDS home 
visit programme run by Ndola Diocese is especially effective in bringing care to people 
who would otherwise receive no care at all. Similarly, the YUHP programme in Nepal is 


bringing health care to people in poor, migrant communities who would otherwise have 
to fend for themselves in the private sector. 
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A concerted effort now needs to be made to assist organisations to develop health- 
related indicators where appropriate, and to improve their own monitoring systems. 
New health information systems do not necessarily need to be created to address this, 

as national systems already exist and can be better applied by CFA partners. As the 
main barriers to the widespread use of the health management information systems 
concern logistics and capacity issues, the partners need more support to introduce and 
use existing national systems. They also need to consider what other data they need 

to collect on a routine basis in order to measure achievement of organisational and 
programmatic objectives, especially with regards to how well they are meeting the needs 
of poor people. 


A further problem that is impacting on quantitative results amongst Cordaid partners 
is the reluctance by some of the Catholic institutions to provide comprehensive sexual 
and reproductive health services. Significant service-delivery gaps were found where 
Catholic diocesan health services have a limited sexual and reproductive health 
programme, particularly where these health services are the only service providers 
available. In both DRC and Zambia, where there are both high rates of maternal 
morbidity and mortality and where communities are heavily affected by the HIV and 
AIDS epidemic, sexual and reproductive health programmes are vital for preventing 
illness and death. 


From a qualitative perspective, health services provided by CFA partners are widely 
appreciated by government stakeholders and beneficiaries. These services are considered 
to be of higher quality than alternative (public) services. In general the quality of care 
provided by different partners can be considered to be technically good or very good. 
Quality is determined by factors such as the quality (and presence) of qualified staff, 
staff attitudes to patients, and levels of material, drugs and equipment available to 
diagnose and treat patients appropriately. All services evaluated rated well in terms of 
material, drugs and equipment in health units. Even where state-run health services 
operate fairly close by faith-based facilities supported by CFAs (as seen in parts of 
Zambia), the faith-based facilities are better able to diagnose and treat patients due to 
having more qualified staff and a better stock of medicines. 


There remain, however, serious challenges to service quality in all three countries. The 
conflict and weak state structures in DRC and Nepal mean there are severe problems 
with recruiting and retaining qualified staff in partner health units. Zambia also faces 
serious problems with retaining staff in remote health units, whether in the state or non- 
state sector, with its problems further compounded by the loss of many trained staff to 
AIDS. The CFAs are working with partners to improve the situation with regards to staff 
retention, for example, supporting incentive schemes and providing trained volunteers. 


New trends in contracting health services between the state and non-state sectors 
provide opportunities for Cordaid and ICCO’s partners to review their relationship with 
the state sector as well as for shaping national health policy. This will also require more 
alignment with nationally agreed quality assurance standards and health monitoring 
system requirements. While acknowledging the very real challenges facing districts and 
dioceses, the long-term sustainability of health services is dependent on their working 
‘closer together. This may require a more open and transparent approach by some 
partners. Similar issues arise where Plan runs health clinics or schools on behalf of the 


government. 
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ed activities are filling a real public health need and benefit poor and 


non-poor alike. For many beneficiaries, though, user fee charges continue ie i 
significant barrier to their ability to use health services, even where costs ave | een 
reduced. The CFAs are not, at present, working with partners or developing policy- 
influencing strategies for reducing the damaging effects of user fees, other than 
supporting small-scale activities with limited reach. Section 8 explores some of the 


mitigating strategies in more detail. 


Community-bas 


Across all countries, financial sustainability of partner activities is extremely weak, and 
will continue to be so for the foreseeable future. This is not surprising in such aid- 
dependent countries, nor in countries such as DRC and Nepal, which have to contend 
with conflict and the threat of conflict. Sustainability of programme activities, especially 
those that are community based, appears to be more certain in those programmes 

that actively engage community members, that encourage self-help initiatives and 

that build community capacity to carry on beyond the life of the project. Examples of 
programmes that have high prospects for longer-term sustainability through community 
mobilisation include BOAD’s work, ELIMU’s literacy programme and the various home- 
based/community-based care initiatives supported by CFAs. Stronger governance 
arrangements that increase accountability by moving from a staff- to stakeholder-based 
model would also improve chances for sustainability, for example as developed by CBRS 
in Nepal. 


Sustainability can also be better assured by integrating programme activities into 
government systems, as is being done with Plan’s programmes and YUHP, set up by 
UMN in Kathmandu. However, as can be seen with government plans to expand the 
ECCD programme in Nepal, public resource constraints may limit the government's 
capacity to implement programmes to the standard set by non-state providers. 


Recommendations on improving quantitative and qualitative impact of health-related 
services 


All CFAs 
e All CFAs should continue to work with and strengthen their community-based 
programmes, as these are having the greatest effect on the health of poor people, and 
have the greatest likelihood for sustainability (e.g. Mama and Papa Bongisa, Kinshasa 
Diocese; Ndola Diocese’s AIDS home-based care; Plan’s outreach work in Zambia and 
Nepal). 
e All the CFAs need to reconsider their own and their partners’ strategies that depend 
on user charges for services they support. User fees are a deterrent to the use of 
services by poor people and appear to contradict the CFAs’ own pro-poor policies; yet 
they also provide much-needed resources to maintain services at local level. A first 
step should be to hold a policy forum that includes staff of the CFAs, representatives 
of their partners/field staff and leading researchers to help frame how the CFAs can 
respond to this and how they can support or influence national policy development 
in either eliminating user fees or creating social safety nets for those who cannot 
afford them. 
Training should be provided to CFA partners (governmental and non-governmental) 
in the collection and use of routine health data to help develop an evidence base 
with regards to health outcomes and ensure it is well linked to the national health 
Management information system. 
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Cordaid and ICCO 

e In order to measure the health impact of CFA support, national strategic frameworks 
should include objectives with measurable indicators that reflect the aggregate 
results of their partners’ own strategic plans and targets. CFA partners also need 
capacity building to help them with developing capacity in this area. This will greatly 
support partner M&E activities, and facilitate future evaluations of health-related 
programmes. 

e Sustainability of health-related services remains a key concern for ICCO and Cordaid 
partners. ICCO and Cordaid should work with their service delivery partners on 
improving the prospects of longer-term sustainability of their activities, through 
designing handover strategies, creating more transparent links with government 
services by making better use of contracting arrangements and building on models of 
good practice for community involvement identified in this, and other, evaluations. 
ICCO and Cordaid should also continue to encourage their partners to seek ways of 
diversifying their funding base. 

e ICCO and Cordaid should examine the model of community capacity building 
employed by Plan and discuss with partners if and how this model could be adapted 
into their community-based work. 


Cordaid 

© Cordaid needs to give serious consideration to how it can support filling the gap 
left by its partners in family planning and other reproductive health services. One 
suggestion is that where the Cordaid (Catholic) partner is the sole service provider 
Cordaid should also fund an alternative non-Catholic partner to ensure these services 
are accessible, or that it use its influence to persuade another donor to provide 
support in this area. At the same time Cordaid should embark on a programme to 
facilitate greater cross-learning between the Catholic organisations it supports and 
negotiate more emphatically for attention to be given to sexual and reproductive 
health within the organisations it supports. 

e Health reforms leading to decentralisation have created a new set of dynamics that 
Cordaid’s partners in Zambia have not fully incorporated into their own thinking. 
Cordaid and its partners also need to reflect on how to build a more transparent and 
trusting relationship with government line agencies in Zambia, including being more 
open about financial flows. 


Plan 

e Plan programmes should consider exploring the UMN experience in Nepal to 
see whether they can further institutionalise their programmes by setting up and 
supporting a local NGO, which would then carry on the main areas of Plan’s 
activities. 


1.2.3. Issues concerning extent and impact of partnerships and networks 

Partnership and networking has had an overall positive impact on the work of the 
partner organisations. The partnerships supported through CFA funding are making a 
meaningful contribution to local and national policies on health for poor people. The 
more established networks, such as CHAZ, CCJDP and CPRS in Zambia and CONAFED in 
DRC are ensuring dialogue both within the networks themselves as well as with national 
‘bodies and other stakeholders. It was reported that building of coalitions with other 
organisations resulted in generation of a greater voice. Several examples of effective 
approaches have been given in previous sections of this summary. 
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erships evaluated also have strong elements of capacity 


building of members and community organisations as key strategies for enhancing the 
work of their networks. The following examples illustrate the importance of capacity 


building for improving the performance of partners: 


Many of the networks and partn 


© CONAFED has developed and runs training modules for members in raising 
awareness of gender issues and supporting the mobilisation of women. 

© ELIMU has developed literacy training modules and management training 
programmes for community-based women’s groups, which directly benefits the 
performance of these groups when applied to their income-generating activities and 
other areas of work. 

e Plan’s programmes in both Nepal and Zambia have worked closely with and 
supported the training of local government counterparts, leading to improved 
performance in such areas as Integrated Management of Child Infections. 

e BNMT’s collaborative work with government staff has built government capacity to 
plan and manage TB programmes. 

e CCJDP and CSPR, through training programmes aimed at members of their own 
networks, have built the capacity and confidence of community-based organisations 
to monitor and hold to account government officials. 


However, as mentioned above, networking and advocacy organisations also need 
support in developing strategic plans, objectives and indicators that will help them to 
set targets and monitor their progress in achieving the purpose of their partnerships or 
advocacy campaigns. Much of the impact of partnerships and networks reported in this 
evaluation is anecdotal, and needs better data to back up the stories of improvements 
provided to the research teams. 


The evaluation also found that there is some confusion as to how the terms ‘partnerships 
and networks’ are being used. In some cases, those organisations designated as a 
network are in fact large local NGOs that provide a conduit for funding to community- 
based organisations (e.g. WOREC in Nepal and BOAD in DRC). These are therefore not 
partnership or network organisations where members are supported by a secretariat. 
However, such large NGOs do provide important umbrella functions, such as capacity 
building, for community organisations that would be difficult to achieve otherwise, and 
therefore enable much greater reach into poor communities. 


Although some efforts have been made, Cordaid and ICCO have not yet been able to 
build effective links between their partners and relevant networks in the countries they 
work in, nor are they playing a role in bringing together like-minded organisations 
within countries, which would have the potential for creating greater synergies. This 
could allow them to scale up innovative practice beyond the scope of their projects. This 
has resulted in some degree of ‘atomisation’ of programmes within countries, with little 


ar of an overall, coherent health strategy that works towards improved population 
ealth. 


For example, the dioceses in both Zambia and DRC operate similar structures and 
within similar systems. However some have been more innovative and responsive 

to community need than others, and yet there seems to be little sharing or spread of 
good, innovative practice. Another example was found with the organisations that 
had branched off from UMN projects in Nepal, where again, there appeared to be 
little sharing of good practice (for example, in terms of governance), which could 
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help organisations to improve their performance. Plan could also create stronger links 
between its various project teams, and between these teams and local/national networks. 


Recommendations for strengthening partnerships/networks 


Cordaid and ICCO 

e Networking and advocacy partners need particular support in developing ways of 
measuring the outcomes of their less tangible activities. The strength and results 
of partnerships can be assessed within the partnerships themselves, using validated 
tools, one of which was introduced in this evaluation. Cordaid and ICCO should also 
support advocacy partners to develop measurable targets and key indicators that will 
help them to benchmark progress towards reaching their advocacy objectives. 

e Once partnerships and networks have undertaken a more robust assessment of their 
partnerships, Cordaid and ICCO should support these organisations to build on their 
members’ strengths so that they become more influential. At present, many of the 
partnerships and networks supported are not creating the synergistic effect expected 
from organisations working together. 

e Cordaid and ICCO already provide some South-South networking opportunities for 
partners and these could be continued and expanded to include a cross section of 
both organisations’ partners where appropriate. Innovative work done by partners 
on gender-based violence should be a key area of expansion as some of the 
organisations working in this area remain small and isolated. 

e In the DRC, Cordaid and ICCO should work together with their faith-based partners 
and other donors to develop a national faith-based health association that can 
advocate on behalf of faith-based health service providers as well as provide a 
framework for quality assurance of health services. It may be helpful to involve 
CHAZ, to draw on its experience as an effective umbrella organisation in Zambia. 


1.2.4 Issues concerning CFA value added 

The CFAs are bringing added value to the work of their partners, primarily through 
flexible and consistent funding and support for capacity building. While there is much 
overlap between the policies of the CFAs, each also has unique elements to its approach. 
Assessment is limited by the lack of identified added value in CFA strategy, in terms of, 
for example, capacity building and human resource development, global North-South 
advocacy, promoting innovation and facilitating linkages and information exchange. 


Examples include Cordaid’s support to strengthening BDOMs and provider partners 

in Zambia in building up their managerial competences, which is contributing to 
making the organisations more efficient and effective, and broadening their funding 
base. Respondents in DRC felt that the presence of a local office for Cordaid facilitates 
interaction between Cordaid and its partners, enhances their participation in donor 
coordination, and facilitates advocacy for improved legislation and policy making at the 
national level. Cordaid’s technical assistants are also filling in human resource gaps in 
hospitals and clinics where it is very difficult to recruit and retain staff, while Cordaid is 
also providing packages of training and incentives to national staff. 


ICCO has a very flexible approach to its funding, allowing it to build partnerships with 
-civil society institutions, while also maintaining links into faith-based organisations 
which are critical health service providers in many countries. ICCO’s approach 

opens opportunities to small organisations that are usually below the international 
development radar. An observed significant added value of ICCO’s support is that it 
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aintain a more holistic approach, which does not 


to adopt and m 
enabled partners P | aspect of development. It was felt that 


alth in isolation but as an integra 
pe i fo given to community capacity building was very valuable though . 
extremely challenging in war-stricken areas. Support rendered by ICCO is strengthening 
civil society in Eastern Congo in terms of providing specialised services as with CME, 
increasing training capacity for community health and creating enabling environments 
(IPASC), and networking with grassroots organisations (BOAD and ELIMU). 


Plan’s work in country brings a more ‘hands on’ approach to community (and civil 
society) capacity building through its community facilitators and through facilitating 
training for health facility staff. Plan Netherlands’ contribution is to provide active 
support for this work, and to promote the child-centred approach. 


Numerous areas of effective innovation were also identified by this evaluation, 

for example, in Plan’s programmes for early childhood care, in the organisations 
supporting women who experienced sexual violence (Cordaid-DRC and ICCO-Nepal), in 
institutionalising new approaches with government and influencing policies on urban 
primary health care (ICCO-Nepal), AIDS outreach and care in Zambia (Cordaid) and 
performance-based contracting in the DRC (Cordaid). 


Some of the partner organisations visited have been involved in a number of capacity- 
building workshops in Zambia and DRC. ICCO has also organised regional level sharing 
meetings for some of its partners, and partners have been enabled to attend regional 
conferences. Some North-South networking is taking place at the global level. However, 
no formal or informal networking and coordination among CFA (mainly ICCO) service 
delivery partners have taken place in Nepal. Networking activities in general are weak 
among all NGOs. While the conflict situation may explain some of this, the fact that 
this evaluation was able to conduct two workshops with health-related partners during 
particularly tense periods shows that it is possible to facilitate greater networking. 


Many partners reflected on the fact that, at least as far as Cordaid and ICCO are 
concerned, they were aware that Netherlands-based programme officers were 
overstretched with the number of countries and organisations they cover. Also, 
frequent changes in staffing within these key link posts made communication very 
difficult, especially regarding any dialogue on organisational strategic changes. To 
many partners, new policy directions outlined in the various CFA policy documents 
were perceived as more of a ‘whim’ of individual programme officers, rather than 
representing important changes with implications for the partnership. 


This evaluation has found that added value could be strengthened. As many of the 
partner organisations funded by the Cordaid and ICCO also receive funding from 
multiple other donors, these two CFAs need to reconsider the niche they can fill to 
increase the added value of their support in line with their global policy objectives. The 
evaluation team identified a number of areas in which the CFAs could strengthen their 
added value, related primarily to supporting partners’ institutional development, such 
as strategic planning, financial management, M&E and communication mechanisms. 
Given their long-standing relationships with their partners, the CFAs also have unique 
Opportunities with which to challenge prevailing norms that continue to exclude women 
and marginalised groups and, especially within the Catholic missions, challenging 
Conservative attitudes to sexual and reproductive health and women’s participation. 
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Recommendations for strengthening added value 


All CFAs 

e As an initial step CFAs should support cross-learning opportunities that bring 
together partners in single countries or regionally to share experiences and discuss 
good practice. These could be done at a sectoral level, though much of the good 
practice observed during the evaluation has cross-sectoral application. This should 
include making more deliberate efforts to link partners involved in running networks 
for advocacy-related work and service delivery partners. 

e CFAs should also develop strategies to strengthen the voice of their southern partners 


in the North, working with partners to agree on priority issues for advocacy by CFA 
policy units. 


Cordaid and ICCO 

e Both Cordaid and ICCO need to elaborate more explicit links, through strategic 
planning processes, between their global policies related to access to health and basic 
services and the programmes they support in countries. These links must delineate 
how the CFAs are adding value to partners’ programmes, including: 

- Challenging prevailing social and organisational norms relating to gender, as well 
as paternalistic attitudes of some partners towards communities they work with. 

- Developing mechanisms for cross-learning between partner organisations to 
ensure the spread of good innovation and promote stronger coherence. This is 
particularly important in DRC and Nepal, where normal coordination mechanisms 
are weak or non-existent and need developing. This could start with an 
exploration with partners of what coordination mechanism would best suit their 
needs and could extend to non-CFA partners in time. 

e CFAs should strive for greater stability of programme officers to allow for more 
consistent and longer-term contacts between partner organisations and the relevant 
CFA. Where this is unlikely, or hard to control, the CFAs need to develop a more 
consistent communication strategy with partners that allows them to engage more in 
dialogue with the CFA organisation concerned. 


Plan 

e Plan should encourage greater cross-learning between staff working on different 
projects, as innovations are not being spread effectively across each country’s 
programme. 

e Plan staff should be provided with further training and support to challenge social 
norms within the CSOs they work with that exclude particular groups, and to use 
their prominent role in the communities to advocate more directly for the rights and 
needs of the most vulnerable. 
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2.1 


2.2 


Introduction and Background 


Background and purpose of the evaluation 


This report summarises the key findings of the evaluation of the health component of 
three of the Dutch Co-Financing Agencies: Cordaid, ICCO and Plan Netherlands. The 
evaluation used a case study approach, with three country programmes selected by 

the three CFAs: DR Congo, Nepal and Zambia. The evaluation was commissioned by 

the three Co-Financing Agencies to contribute to the wider, ongoing Co-Financing 
Programme evaluation (2003-2006), which includes studies undertaken on six additional 
themes. ICCO had lead responsibility for coordinating the health component evaluation. 


Cordaid, ICCO and Plan Netherlands agreed to commission a programme evaluation 

to gain better insight into the results of their support of health (related) activities for 
poverty reduction and to learn from their practice. The evaluation is seen in relation to 
the ongoing evaluation efforts related to the CFP (Co-Funding Programme) that have 
been undertaken since 1980. Based on a review of the results of previous evaluations 
and changing organisational policies, the three organisations decided the present 
evaluation must give attention to both direct service delivery and to civil society 
building aspects of the CFA-supported interventions in the health sector. 


This report summarises the results of the three country evaluations and analyses these in 
relation to the stated polices of the three CFAs as well as the key variables used within 
this evaluation. Please see Appendix 2 for the Terms of Reference for the evaluation, 
and appendices 2 through 4 for the detailed evaluation reports in see separate country 
reports for the detailed evaluation reports. 


National contexts 


The three countries selected for inclusion in the evaluation, Democratic Republic of 
Congo (DRC), Nepal and Zambia, all feature in the low end of the UNDP’s Human 
Development Index, with the majority of the population in each country living below 
the World Bank’s absolute poverty line of US$1 per day. Two of the countries (DRC and 
Nepal) are experiencing ongoing conflict, which has had a critical impact on people’s 
livelihoods and personal security, as well as severely disrupted public service delivery. 
Although enjoying political stability, Zambia is facing the triple threat of poverty, 
continuing food security difficulties, and the exacerbating effect of its severe HIV/AIDS 
epidemic. While large parts of the population in all three countries are highly dependent 
on external aid in order to maintain a basic standard of living, the dynamics of conflict 
have had the dual effect of making people both more vulnerable and also less easy to 
reach through development assistance. 


‘The tendency in conflict situations (and often in highly aid-dependent communities) 


is to import technical assistance and invalidate local experience and expertise. The 
imperative to deliver assistance tends to overrule the importance of continuing to 
build on local capacity and to support local coping strategies. Each of the three CFAs 
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evaluated in this report have responded to the challenges that the national pene 
present them in a variety of ways. All emphasise local capacity building as part O 

their development strategy, whether their partners are working within a complex = 
humanitarian crisis or a more stable development situation. The results and analysis in 


this report are presented with these contextual issues in mind. 


2.3 Health-related services 


Health systems are made up of a variety of actors, institutions, funding and service 
delivery mechanisms. The fundamental system features of health service delivery are 
how it is provided and how it is financed. In all three countries that took part in this 
evaluation, the health systems are made up of a combination of state and non-state 
providers. In terms of health-related services for poor people, the main providers are 
the state, NGOs and faith-based organisations. However, the countries also have a 
significant informal private sector made up of traditional healers, as well as ‘brief case’ 
or ‘quack’ doctors who are unqualified individuals with ready access to drugs. These 
informal providers often give much better access to treatment for poorer and more 
remote communities than the more formal health services can. The formal for-profit 
private sector in all three countries caters primarily for the wealthier (urban-based) elite, 
who can pay for the more specialist services offered. 


Historically, faith-based clinics and NGO clinics were set up in remote areas of countries, 
with an ethos of service to marginalised communities. With their direct connections 

to external sources of funding and arguably better management, faith-based and NGO 
clinics have been seen as better resourced and more able to cater for the users of their 
services than government services. However, government-run services are seen as the 
backbone of the health system, and the populations of most countries consider that 
governments have a duty to ensure access to basic health care for all, a duty that cannot 
be imposed on non-state providers by communities themselves. Government services 
have tended to be very poorly resourced and understaffed. Both government and faith- 
based clinics have substantial reach across the three countries studied here. 


In Zambia and Nepal, efforts have been made to coordinate and regulate services across 
the state and non-state divide through reforms to the health sector. Memorandums of 
understanding guide the relationship between the state and faith-based organisations 

in both these countries. Such efforts are at a very early stage in the DRC, where 
government policy and influence in the health sector has been extremely weak for 

the last 30 years. In the DRC an EC funded programme has introduced government 
contracting whereby non-state providers enter into a performance-based contract with 
the government to deliver an agreed set of services. Contracting has gained increasing 
interest amongst the donor community as a means of supporting government regulation 
and coordination of the health system in weak states, while recognising that non-state 
organisations are better placed to deliver services and programmes. 


In all three countries health services are funded through government budgets, donor 
funds and charges to users. Very limited social insurance schemes exist in the three 
countries, and these tend to be geared to urban and better-off populations. All countries 
have exemption policies aimed at ensuring that the poorest and most vulnerable parts 
ay ~ population are not charged for services. In no country are these exemption 
policies adhered to or implemented consistently. This situation mirrors findings from 
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more in-depth research, which indicate that user fee exemption strategies are simply 
not implemented and are often abused.? Some small-scale efforts have been put in place 
to make health care more affordable to poor people in urban and rural areas, primarily 
through non-governmental projects. These have included ‘associations mutuelles’ 
(which are member based and act as local insurance schemes), cash transfer schemes 
(where money is given to poor families to help pay for school fees and health care costs, 
usually through social services) and barter systems (where the equivalent of fees are 
paid through commodities, such as maize or other locally produced goods). 


2.4 Evaluation aim and questions 


The aim of this evaluation is to: gain insight into the contribution of the CFAs and their 
partner organisations to the improvement of sustainable use of, access to and control 
over health-related services for the poorer sections of the population, with a special 
emphasis on women and youth. 


Questions to be answered by this evaluation are provided below, with sub-questions/ 
areas of enquiry: 


e Which visions, strategies and activities do partner organisations have to improve 
access to, use of and control over basic health (related) services by (potential) users in 
the programmes funded entirely or partially by the CFAs? 


© What are the qualitative and quantitative results of these programmes and for whom 
do they improve sustainable access to, use of and control over basic health (related) 
services? 


© To what extent do (complementary) partnerships, participation in networks and/or 
alliances contribute to better performance of the partner organisations and the health 
system in general? 


e With what results do (complementary) partnerships, participation in networks and / 
or alliances contribute to a better performance of the partner organisations and the 
health system in general? 


© What is the added value, in terms of partnerships and results, of the CFAs’ support 
for the partner organisations and their programmes? 


The above questions were answered using an analytical framework that includes six 
variables that the evaluation team considered to be core to the questions that needed 
answering. As this evaluation seeks to understand both outcomes and processes, 

the variables to be assessed are as follows: Impact/effectiveness, Quality, Coherence, 
Participation, Sustainability, Added value. As stated in the Technical Proposal (p. 

9, annexe), ‘Programme outcomes are delivered or mediated through a number of 
processes or outputs, including direct service provision to poor communities, civil 
society building through development of alliances and networks, citizen and client 


2 Creese AL (1991) User charges for health care: a review of recent experience. Health Policy and Planning 
6: 309-319; Gilson L (1997) The lessons of user fee experience in Africa. Health Policy and Planning 12: 


273-285. 


often undertaken by the intermediary partner 
by the CFAs are the key strategies 
n each country and b) facilitation 


voice and policy-advocacy activities, 
organisations. Investments/inputs to these processes 
of a) working in partnership with a number of groups i 
of organisational development or capacity building’ 
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3.3 


Methodology of Evaluation 


Overview 


The methodology is based on well-tested case study methods used in similar styles of 
research. Both Marsden et al. (1994) and Roche (1999) either reviewed or used the case 
study approach as a means of evaluating the impact of development assistance. The 
methodology in this study is based on the learning and best practice advice outlined by 
both groups of authors. Standardisation of data collection was assured through use of 
4 common framework across all three countries (see TORs in Appendix 2) and common 
data collection tools (see below). 


As outlined above, the evaluation examined the CFA programmes in three different 
countries: Democratic Republic of Congo, Nepal and Zambia. The three CFAs formulated 
the following criteria for the selection of the case study countries: 


Substantial financial CFA contribution to health-related programmes 
Considerable number of partner organisations active in the health (related) sector 

e Existence of a network, support organisation or other formal or informal structure in 
the health system in which the partner organisations participate in order to undertake 
lobbying, advocacy or other activities or receive support from it 

e Evenly spread over the continents 

e Have had no recent evaluations 


Research teams 


In line with the philosophy and practice of the CFAs, the evaluation study in each 
country was undertaken by national teams, with oversight provided by two international 
consultants. The teams were recruited through the HLSP Institute's existing networks, 
with care taken to ensure that members of each team had no conflict of interest and 
could therefore provide an objective overview, e.g. none had worked for any of the 
programmes selected for evaluation. 


Definition of terms 


A number of terms were used in the evaluation questions and in the variables 
considered during the evaluation. The research teams understood these terms as follows: 


‘Health related’: Activities that lead to direct health improvement, including 
health promotion, disease prevention (e.g. water and sanitation), 
clinical treatment and care; as well as activities whose main 
purpose is to improve health, e.g. health advocacy and health 
systems support. 
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3.4 


‘Partnership’: 


‘Impact/Effectiveness'’: 


‘Quality’: 


‘Coherence’ 


‘Participation’ 


‘Sustainability’ 


‘Added value’ 


Sampling strategies 


the formal 
The primary partnerships evaluated were 
partnerships and networks receiving support from the CFAs. The 
other level of partnership examined was between the CFAs and 
the organisations they fund and support in the three countries. 


Achievement or likely achievement of objectives 
Improvement in programme health indicators 
Improvement in organisational development indicators 
Improvement in advocacy indicators 


Technical quality of the design of programmes 

Quality of personnel - qualifications and professional 
development 

Quality of services - adequate and consistent supplies of drugs 
and equipment 

User views on quality of services (acceptability and satisfaction) 


With national health policies and standards 

With CFA health policies 

With Dutch government policies in the three countries 
Potential for innovations to influence the development of 
improved policies and strategies of both national governments 
and of CFAs 


Full involvement of beneficiaries and/or partner organisations 
in planning, managing and monitoring 

Full and equal involvement of normally disadvantaged groups 
(women, youth and ethnic minorities/low caste) in planning, 
managing and monitoring programme activities 


Financial, institutional and human capacity to continue beyond 
programme support 

Of expected outputs and outcomes of programme interventions 
Of partner organisations themselves 


The benefit of receiving CFA support beyond funding 

e To individual organisations and partnership/networks 

¢ Complementarity of the relationship 

e The benefit to individual organisations of membership in a 
network or partnership 


Sampling of partners in countries 
Selection of the specific partners to be involved in the study was based on the following 


criteria: 


* Aims and objectives of the partner organisation and/or the health interventions 
° Level of activity of the partner in the health field 
* Balancing rural and urban interventions 
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e Mixture of different types of organisations and interventions (direct service delivery, 
advocacy, community health promotion and empowerment etc.) 


Gender is a cross-cutting theme of all the CFAs and featured within the analysis of 
individual partner programmes. 


In order to reflect the change from a needs-based approach to a rights-based approach, 
whereby the poor are empowered to claim their rights, other organisations beyond the 
health sector were included. In this respect the teams looked at the country level at links 
with networks with strong judicial aspects, civil society networks advocating for and 
monitoring health within PRSPs, women’s networks that promote health improvements 
and, in the case of Congo, also networks that address violence issues. 


In order to make the most objective selection of partners possible, each CFA was 
asked to fill in a partner information sheet for any partner that was being funded to 
implement a health-related component within their organisation (see Appendix 5). 
One of the international evaluators also interviewed the relevant programme officers 
in each organisation to gain a better insight into both the overall programme strategy 
for the country as well as the individual partners and programmes. Once the teams 
began working within the countries adjustments had to be made to which partners 
were selected based on security concerns (especially in Nepal and DRC) and logistical 
concerns (e.g. physical and financial access for visits). For this reason DvDW in Nepal 
had to be dropped from the initial list as the team could not reach their area of work. 


Using the criteria outlined above a selection of partners was made in each country, as 
set out in Table 1. 


Table 1 - Partners selected to participate in evaluation 


DRC Nepal Zambia 
Cordaid ICCO Cordaid 
Bureau Diocesan d’Oeuvre Medicale United Ministries of Nepal (UMN) Catholic Centre for Justice, 
(BDOM) Kinshasa, Kananga and - Yala Urban Health Project (YUPH) Development and Peace 
Bukavu - Centre for Community Development Christian Health Association of 
Ecole de Sante Publique (ESP) (CCDN) - Makwanpur Zambia 
Comite National Feme et - Sakriya Sewa Samaj (SSS) Civil Society for Poverty Reduction 
Development (CONAFED) British Nepal Medical Trust (BNMT) Diocese of Ndola 
SPS (Cordaid assistance CVICT Diocese of Mpika 
discontinued) WOREC Diocese of Mansa 
Centre Olame Community Based Rehabilitation Services 
(CBRS) 
DRC Nepal Zambia 
IcCcO Plan Plan 
Bureau Oecumenique d’Appui au Evaluated projects in Evaluated projects in 
Developpement (BOAD) - Makwanpur - Mazubika 
Centre Medicale Ecumenique (CME) - Kathmandu - Chibombo 
Beni 
ELIMU 
Institut Panafricain de Sante 
Communitaire (IPASC ) 


(Partner details are provided under individual country sections of this report. See 
separate country reports). 
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Sampling of health data | 
Partners were asked to provide baseline and monito 
assessing progress within their projects. The teams a 
local area health data with which to compare project data. 


ring data that they collect for 
Iso tried to collect national and 


Sampling of key stakeholders i 
Interviews were held with government staff from relevant departments (national and 


local), service provider staff and beneficiaries. At the inception workshops partners were 
asked to provide a list of whom they thought were the key people to be interviewed 
about their projects (e.g. their contact persons in Ministries of Health.) Beneficiaries 

to be interviewed were selected through purposeful sampling in order to ensure 
representation of views of women, men and youth. Organisations being evaluated 
provided initial introductions into communities, which the teams then followed up with 
invitations to be part of a focus group discussion. | 


3.5 Data collection methods 


International level 

The international consultants reviewed policy and programme documents of the three 
organisations in order to understand better the policy context within which they were 
funding partners and programmes in the three case study countries. Particular emphasis 
was given to health policy documentation, and to any documents referring to country- 
specific policies and strategies. These were then summarised to inform both the country- 
specific evaluations as well as the overall findings for this report. 


Country level 

Standardisation of data collection across countries was assured through development 
of interview guides and focus group protocols (see Appendix 7), though teams 

were advised to adapt these to local circumstances. Organisational and partnership 
assessments were carried out using a validated self-assessment form, either filled in 
individually or as a group. Research team members then discussed why individuals and 
groups had rated their organisations the way they did (see Appendix 8 for tool used). 


Each research team was tasked with carrying out the following evaluation activities: 


¢ Inception workshop - Research teams were asked to invite partners to an inception 
workshop so that the evaluation process could be explained, their questions could be 
answered and they could have input into the process more generally by suggesting 
any key issues they wished to see covered. 

Health data collection - In order to evaluate the possible impact of projects and 
programmes, research teams collected project health data (service statistics and 
disease data) and compared these with local area and national statistics. 
Organisational assessments - An organisational assessment tool was also provided 
to assist with evaluating organisation-related variables (see Appendix 8). The 
organisational assessment tool was piloted during two inception workshops (in 
Zambia and Nepal), both to assess the acceptability of using the tool during the 
evaluation, as well as to test its applicability. In both cases participants found the 
tool to be a useful addition and agreed to its use in their organisations. 
Organisation/project staff interviews - Partner organisation and project staff were 
asked to provide their perspectives on how well their health-related work was 
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progressing towards achieving their objectives as well as to give views on the added 
value of the partnership with the CFAs. 

e Key stakeholder interviews - Stakeholders in each country who were not directly 
involved in partner programme implementation were asked for their views on 
the Dutch CFA health programmes. These stakeholders included staff in national 
ministries and the relevant staff in Dutch embassies (where appropriate) amongst 
others. 

e Beneficiary interviews - focus group discussions and individual interviews were 
carried out with beneficiaries to explore how health-related activities funded by 
the CFAs may have improved their access to, use of and control over health-related 
services. 

e Feedback workshops - Once data collection had been completed and an initial 
analysis developed, partners participating in the evaluation were invited to attend a 
feedback workshop so that they could hear the results from the research team, fill in 
any further gaps in information and correct any misconceptions that arose from the 
evaluation. 

e Interviews and correspondence with key CFA staff. 


3.6 Limitations and constraints 


Data limitations: Most of the organisations covered in this evaluation, including ICCO 
and Cordaid and their partners in-country, have not developed measurable indicators 
against which to monitor progress towards achieving objectives, rendering attempts 
to measure impact difficult. Efforts were made to compare outputs with local area 
and national health indicators, but this also proved methodologically difficult, due 

to problems with identifying the denominator and the generally poor levels of data 
available. 


Methodological constraints: In order to ensure reliability and validity within the 
evaluation process, the same methods were used across all three countries. While this 
worked to some extent, the rigidity imposed by such standardisation led to problems 
particularly in the DRC component of the evaluation. Other methodological constraints 
related to this evaluation being asked to measure health impact, which requires far more 
in-depth evaluation resources than the time and finances allocated. 


Generalisation concerns: Case study methodology is limited in its ability to generalise 
findings to the whole of any organisation’s programme. The findings presented in this 
report relate to the situation in the countries covered by the evaluation. However, the 
analysis examines these findings in the light of CFA policies and general trends in 
development aid. To generalise these findings it may be necessary to test the analysis 
with partners in other, hon-case study, countries. 


Logistic concerns: Problems impeded certain aspects of the evaluation process. As 
participating partners were not chosen until after the tendering process and budget had 
been completed the time and distance to be covered to reach certain partners imposed 
considerable constraints on the research teams. The planned inception workshop could 
‘not be held with Eastern-based partners in the DRC due to communication and logistical 
problems. However, feedback workshops did occur in all three countries and were very 
helpful to the research teams. On reflection, the tender documents could have provided 
more detail about the numbers and location of potential partners to be evaluated, as 
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d number of organisations to be included from each country. This 


well as a propose 
and planning. 


would have helped ensure better tailored budgeting 


oncern the teams had to cope with. The security situation 
the ability of research teams to either reach certain 
ry groups. As a result, certain partners were 
r were asked to meet with the research team in a 


Security issues were another c 
in DRC and Nepal also limited 
project areas or to interview beneficia 
dropped from the original list chosen 0 
location that the team could access more easily. 


Time constraints: Related to the above, all the research teams felt seriously hampered by 
time constraints. On reflection, the scope and scale of the work required and expected 
by the CFAs was greater than the time and funding permitted. As a result, some aspects 
of the programmes evaluated could only be looked at superficially, especially with 
regards to getting a broad spectrum of beneficiary views. 


3? 


4.1 


4.2 


Findings from CFA Document Review 


Overview 


A number of policy documents were reviewed for this evaluation, both for the overall 
Co-Financing Programme and from the individual organisations themselves (see 
Appendix 6 for a complete list). From the overall CFP (or ‘MFP-breed’) documents 

it is clear that the policy framework of the Netherlands CFP aims at promoting 
structural poverty reduction in the South and in the poorest countries of Central and 
Eastern Europe, while also working to achieve universal human rights. All of the Co- 
Financing Agencies (CFAs) must align their own programmes to this overarching policy 
framework. The six CFAs are allocated core funding from the Netherlands government 
on the basis of their business plans, with total co-funding equalling approximately 11% 
of total Dutch ODA. 


The Co-Financing Programme defines structural poverty reduction as improving 

living conditions and building social structures, thus empowering the poor and future 
generations to adopt a sustainable and humane lifestyle. Intervention strategies outlined 
in the policy include: 


e Direct poverty reduction 
e Civil society development 
e Lobbying and advocacy 


Structural poverty reduction thus includes direct support to services with sustainable 
improvement of social relations and aims at fair distribution, economic growth, 
democratisation and ecological sustainability. Civil society should provide the frame- 
work for citizens to claim their rights and to demand democracy. 


The Co-Financing Programme Advisory Committee also underlined in 2003 the 
importance of the role CFAs play in developing civil society capacity and supporting 
structural poverty reduction. As such it has recommended that criteria should focus 
more on the quality of the management of civil society organisations rather than on 
the character of the organisations themselves, as well as that CFAs be less restrictive in 


terms of what they fund and where their programmes are. 
Overview of Cordaid, ICCO and Plan general and health-related policies and 
strategies, partner policies and gender policies 


Table 2 provides an overview of the general and health-specific policies, strategies and 
activities of Cordaid, ICCO and Plan. As can be seen in this table, each organisation has 


adopted a slightly different approach to aligning their own policies to those of the CFP. 


Partner policies 
Cordaid has shifted from project funding to strengthening social organisations (Partner 


Focus Policy). These are intermediary or service organisations, network or umbrella 
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organisations working for the poorest sections in society and national capacity-building 
ards technical assistance is changing from providing 


organisations. Cordaid’s policy tow 7 ne 
long-term external assistance to building local capacity and human resources. Cordaid’s 


code of conduct increases accountability towards partners. 


ICCO’s partner relations can be based on funding programmatic issues, or strengthening 
institutional and strategic capacity. As a result of the changing focus from a needs- 
based to a rights-based approach, ICCO’s partners are increasingly being selected from 
an empowerment perspective, whereby short-term service activities are linked with more 
structural approaches (with a mix of different partners). 


Plan’s partnerships are currently being discussed with a view to more effectively pooling 
knowledge and strengthening lobbying and advocacy. Capacity building focuses on 
organisational aspects, networking and building leadership in CBOs and the country offices. 


Gender 
Gender is a clear criterion in Cordaid programmes, and Cordaid recognises that in 


certain countries, like in DRC, there is a need to strengthen the capacity of its partner 
organisations in this respect, especially where it concerns reproductive health and HIV/ 
AIDS programmes. Cordaid has a tool to help with improving the gender orientation of 
its programmes, but recognises that it is not working very well. 


Gender is represented in each of ICCO’s policy themes, with a focus on women’s rights, 
access of girls and women to basic services and women’s access to trade and markets. 
Counterpart programmes must include a policy on gender. ICCO has an instrument to 
assess gender issues when analysing applications from organisations. However, there is 
no tool for supporting partners to improve their own gender-sensitive strategies in line 
with ICCO’s policies. 


Plan’s strategic priorities for gender are gender mainstreaming, personal safety and 
security (child abuse, prostitution, trafficking, HIV/AIDS) through developing capacity in 
Plan to address and institutionalise gender aspects, and to increase the participation of 
girls and women in community development. 


Youth 
The only CFA out of the three that focuses on children and youth is Plan. Plan’s policy 


documents have the theme of a child-centred approach interwoven with all its policy 
themes and strategies. 


Health 


The three CFAs have broadly similar health objectives, and employ similar strategies, as 
outlined within their organisation’s health policies. 


Cordaid’s health policy aims to improve the health of poor and vulnerable people 
through a combination of organisations (primarily Catholic dioceses) that are involved 
in direct health service provision, community-based programmes (whose focus is more 
on health promotion) and care for the chronically ill, especially people living with AIDS. 
ICCO does not have a written health policy per se. The focus of its health-related work 
Is to improve access to health services and health-related services in the countries where 
It supports such programmes. Its Strategies include supporting direct service provision 
through faith-based organisations, as well as support to national NGOs working on 
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community health-related activities. ICCO also supports advocacy organisations and 
networks, whose work has an effect on health improvement initiatives. 


The focus of Plan’s health-related programmes is on improving the health of 

children and their families, especially mothers. Plan provides direct health service 
delivery, through running its clinics or giving support to public facilities. Plan has 

also incorporated community-based health work in its Early Childhood Care and 
Development centres and funds other public health-related activities, such as improved 
water and sanitation, as well as improved nutrition programmes. 


4.3 Similarities and differences in health programme support 


All three CFAs are learning organisations and assume an active role in working in 
international alliances and in building stronger relations between civil, bi-lateral and 
multi-lateral stakeholders. Although the individual CFAs in certain circumstances 
collaborate with each other at regional and country level (emergency aid in Eastern 
Congo for example) and in several working groups as ‘Evaluation and Policy’ and 
‘Gender’ they mainly attune to their own international networks and have their specific 
constituencies and partner organisations. 


The three CFAs evaluated recognise that direct poverty reduction, civil society building 
and advocacy should reinforce each other, thereby changing their paradigm from 

a needs-based to a rights-based approach. In ICCO’s policy theme ‘access to basic 
services’ this shift implies matching programmes that serve to fulfil the immediate 
needs of people with a more long-term process aimed at strengthening their capacity in 
claiming rights. Plan is changing its emphasis from a child-focused approach towards 
4 child-centred approach, with child rights as a central theme in setting priorities for 
programmes and policies, and putting emphasis on active involvement of children 

and youth in decision-making processes. Cordaid sees improvement in health not only 
as increasing access to health services, but also as strengthening participation of the 
target population in the management of and control over health care resources and 
programmes 


Cordaid, ICCO and Plan, each in their own way, encourage innovation in products, 
strategies and operations. Improving knowledge management is a new and indispensable 
challenge for Plan-N. Plan-Int and Plan-N are systematically improving analysis and 
follow up of evaluations and other internal studies, with linking and learning as the 
prime objective. Plan-N will in certain areas where it has a comparative advantage 
build specific knowledge centres. Cordaid will develop further to become a leading 
organisation in the field of private development cooperation and envisages responding 
in an alert way to current developments and new insights. An example of innovation is 
the performance-oriented contractual approach that Cordaid’s partners use to monitor 
performance of clinics. For ICCO the changing focus on rights-based processes involves 
training and strategy building sessions at regional or national level, follow-up sessions, 
drawing up civil society reports to be submitted to regional or UN Human Rights 
committees, working with UN special reporters, preparing toolkits or training materials, 
‘and promoting regional networks, and training and resource centres for economic, 
social and cultural (ESC) rights. 


-DecemBeR 2004 
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Table 3 summarises financial flows from the three CFA organisations, both in terms of 
total expenditures for 2002-2004 as well as percentages related to funding from the CFP 
and specifically to the three countries studied in this evaluation. Cordaid spends a higher 
amount on health overall, as well as a higher percentage of its total budget on health- 
related programmes. Plan is the next highest spender on health with ICCO putting the 
least percentage of its total funding towards health-related activities globally. 


ICCO’s and Cordaid’s policies recognise that co-operation on an equal basis with 
partner agencies in the South is imperative in effectively reducing poverty. Partners 
are increasingly claiming their autonomous role in society and in so doing influencing 
the agenda. Therefore for both organisations the relationship with partners has been 
reviewed, with an increasing tendency to provide organisational and institutional 
support with a focus on improving efficiency and quality. Both organisations have 
laid down their own role and responsibilities in a code of conduct. Dialogue is being 
enhanced with partner organisations about their vision, mission, how to evolve 
towards strategic planning and how to improve managerial and financial capacity. In 
this respect, the two organisations use both the instruments of financial support and 
technical assistance and advisory services. 


In view of its position in the international setting of Plan, Plan-N funds projects, which 
are proposed by the Plan Country Offices rather than organisations (in contrast to the 
operation of the other two CFAs included in this evaluation). These projects are mainly 
implemented by Plan staff, CBOs and local NGOs, whereas ICCO’s and Cordaid’s partners 
are mainly intermediary or service delivery organisations and network or umbrella 
organisations. Cordaid’s main service delivery partner is the Catholic Church in the 
countries where it works, while ICCO works with both Protestant missions and local 
NGOs. For Plan local governments are also considered as partners, which is not the case 
for ICCO and Cordaid, though some of ICCO and Cordaid’s partners are themselves key 
partners for national and local governments in the three countries. Plan’s programmes 
have a stronger emphasis on capacity building of local government staff, while ICCO 
and Cordaid focus on capacity building of their partner organisations. 


With regards to cohesiveness of policy and programme, Plan’s policies and programmes 
are the most directly linked to each other. This is not surprising as Plan is an operational 
organisation with connections from international level through to community level 
staff. Cordaid has maintained cohesiveness as it has defined country level policies and 
through working with one primary partner in developing countries, the Catholic Church. 
ICCO has perhaps the weakest cohesiveness, with country policies remaining vague, 
with no clear objectives or indicators laying out what ICCO hopes to achieve through its 
programme. 


Priorities of the three CFAs are closely linked to the MDGs: alleviation of poverty and 
hunger; global partnership for development; equality of women and men; improving 
basic health care with special attention to reducing child mortality, improving maternal 
health, increasing access to affordable medicines and combating HIV/AIDS; sustainable 
management of natural resources, particularly by cultural minorities; and improving the 
circumstances of young people. 


For all three CFAs gender is a cross-cutting issue, although capacity to mainstream and 
monitor gender activities needs strengthening. Plan-N also emphasises children’s rights 
and advocates greater involvement of children and youth in their programmes. 


-DecemBerR 2004 
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5.1 


Findings: Democratic Republic of Congo 


Country context 


The Democratic Republic of Congo (DRC) has one of the most challenging profiles, 
socially, politically and economically, in sub-Saharan Africa. Having endured over three 
decades of ‘kleptocracy’ under Mobutu Sese Seko’s presidency, the country has been 

in the throes of civil war since 1996. It is estimated that almost 4 million people have 
died in the DRC as a result of the conflict there. Despite this, both the conflict and the 
suffering it has caused have gone relatively unnoticed by the international community. 
The worst-affected provinces in the country have been the eastern provinces, especially 
North and South Kivu. 


Table 4 - Key health data for DRC 


Infant Mortality Rate 177/1000 
Under Five Child Mortality Rate 205/1000 
Maternal Mortality Rate 990/100,000 
Life expectancy 41.4 years 
Total Fertility Rate 6.7 
HIV/AIDS prevalence 5% 
Moderate malnutrition 1 to 4 13.2% 
Severe malnutrition 1 to 4 10.7% 
<5 fully immunised against TB 55% 
<5 fully immunised against measles 45% 


* from UNDP (2004) Human Development Report 2004 


Even though health statistics in the DRC are of questionable validity, due to problems 
with data collection, Table 4 shows that the infant and maternal mortality rates in DRC 
are very high. It is likely that these rates are even worse than current health statistics 
are showing, as various humanitarian organisation reports for highly vulnerable groups 
have calculated estimated mortality rates at over 200/ 1000. 


In the 1970s donors and NGOs worked with the then Zaire Government to set up what 
was one of the first decentralised health systems in the world. ‘Zones de sante’ were 
created for the entire country and then essentially divided up between donors and health 
service provision organisations. Central government support was non-existent and 
external funding for health services was augmented by user fees and payment for drugs. 


Today, health services continue to be highly dependent on external funding and 
on non-state provision, including the churches. The government has, with support 
primarily from the European Commission, begun to assume a stronger coordination role, 


‘accepting that it will be a long time before it will be able to provide adequate services 


through a government system. (See DRC country report, section 1 for more detail on the 
national context). 
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5.2 CFA country policies and partners 


5 - Cordaid and ICCO health policy and programmes in DRC 


Cordaid 


1 Community-based health car 
service provision 
a Ensuring package of essential curative and 
preventive quality care 
b Strengthening organisational capacity of 
health services 
c¢ Increasing participation of target groups in 
management of health care 
2 Improving reproductive health care 
a. Increasing awareness about sexual and 
reproductive health 
b Increasing participation in managing 
reproductive health 
c Participation in lobbying and advocacy for 
reproductive rights 
3 HIV/AIDS 
a Prevention, treatment and care 
b Mainstreaming 


Table 


e and basic health | There is no country strategic framework for ICCO's 
programme in DRC. 


Policies/Objectives 


Basic services constitute the largest sector, of 
which approximately 50% is health, including 
water and sanitation in North and South Kivu. 


Mainly due to limited impact on poverty 
reduction, over the past 10 years focus of 
collaboration is shifting from church-based to 
secular organisations, with a more geographical 
concentration. In the course of 1994, partners in 
the eastern part of the country became a priority, 
which resulted in a downward trend in overall 
expenditure for the country. The war brought 
expenditure further down to 50%. Starting in 2000 
remarkable recovery occurred, leading to cash flow 
problems; funds for Burundi were reallocated to 
close funding gaps in Eastern Congo. 


Health integrated into access to basic services. 
There is a potential role in improving quality of 
services through IPASC and CME. 


Organisations supported by Cordaid work 
within strategy of DRC zone de sante and are 

actively advocating for poverty reduction and 
gender equality. 


DRC government 
policy and strategy 


Fit with Dutch The evaluation team was not able to meet with Dutch Embassy officials 
- DRC government 


policy and strategy 


BDOMs: Medical arm of the Catholic Church 
in the DRC, providing health services, 
community-based health promotion and in 
some cases managing ‘zones de sante’. 
BDOM Kinshasa 

BDOM Kananga 

BDOM Bukavu 


Partners included 
in this evaluation 


IPASC: Primary health care training institution 
located in the far north-eastern corner of DRC. 
Provides training and support for auxiliary nurse 
level staff and registered nurses, runs community- 
based health programmes and carries out research 
activities. 


CME: Supports 7 evangelical organisations working 
in and around Beni, to run hospital services, 
ensure constant drug supplies, runs training 
programmes and operates community-based health 
programmes. 


CONAFED: Works at a national level to promote 
gender equality. Has branches at provincial level, 

runs awareness-raising and training modules with 
its members, and promotes the leadership role of 
women in organisations. 


ELIMU: Works in Uvira and Fizi supporting the set 

up and development of community-based women's 
organisations. Runs literacy training and training in 
income-generating activities. 


ESP: Works within the Ministry of Education and 
is part of the University of Kinshasa, training 
public health professionals. 


BOAD: Supports 9 organisations in and around 

Goma, which assist 180 CSOs in providing material 
assistance and training to communities affected by 
conflict. 


Centre Olame: Provides psychological and social 
support to traumatised women and in particular 
those who have suffered from sexual violence. A 
specific arm of the BDOM in Bukavu. 


Total 2004 health-related funding = 
€ 1,703,061, € 949,952 from CFP. 


Total 2004 health-related funding = € 327.505, 
with € 327,505 from CFP. 


27 partners total 


4 partners 3 strictly health 
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Finoincs: Democratic Repusic of CONGO 


5.3 Results 


5.3.1 Impact/effectiveness 

Table 6 shows that service delivery organisations in DRC are seeing substantial numbers 
of patients. While the figures provided are for 2003, these should be taken as averages 
of the level of service provision activity as there is little variation between these and the 
previous two years (2001, 2002). It must be stressed, though, that the faith-based service 
providers in DRC are the only health care providers in their area. The DRC has long 

had a policy of contracting out health services and health management to non-state 
providers. The faith-based organisations supported by the CFAs are amongst the most 
important non-state providers in the country, without whom the health system would 
collapse. Therefore one measure of their effectiveness would be to consider what health 
care would be available if the CFA partners stopped providing services. The answer is 
that there would be no health care within entire regions of DRC. 


This being said, quantitative data for DRC, at national and local level, as well as from 
project partners is extremely difficult to obtain for a variety of reasons. These include 
unreliable national data due to data collection problems, lack of standardisation of data 
collected by different service providers, as well as no baseline data (or lost data due to 
the conflict) within the projects or organisations. There is also almost no data on the 
situation of the general public in the areas where the CFA partners intervene. As Tables 
5 and 6 indicate, there is no true picture of what the population base is for providing 
service use rates, with percentages rising well above 100%. This has made quantifying 
and comparing the impact of ICCO and Cordaid’s partners very difficult. 


Neither Cordaid nor ICCO have quantitative health-related indicators that allow 
measurement of progress against health-related objectives in the DRC. While Cordaid 
has a country programme plan, ICCO has no strategic framework for DRC. Therefore, 
both quantitative, and to some extent qualitative measures of impact are reliant on the 
partners’ own strategies and targets. Key indicators for impact are the immunisation 
rates obtained by Cordaid service provider partners, which are substantially higher than 
national rates. These are presented in Table 7. 


Table 7 - Comparison of national and project level data-immunisation 


[seemed celal) Oop OM varisnasa\"}; UBDOMINSnangE BDOM Bukavu 
cowcecc) | om | 85% 


sai. sea i E 


There are also no clear indications that CFA inputs into individual organisations, as 
opposed to those of other funding partners of those organisations, have led to any 
greater or lesser impact on improving access to, use oF control over health services. As 
can be seen in Table 5, most organisations are recipients of funds from multiple donors, 
all of whom have some influence over partner organisation policies and programmes. 
While the CFA funds are seen as more flexible than those of other donors it does make 
it even more difficult to ascertain how important the financial (and even non-financial) 
contributions are to the overall strengths or weaknesses of partner organisations. 

It is also hard to analyse the impact of the length of partnership between the CFAs 
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as most of the longer-term relationships are with well 


d individual organisations, 
ie multiple partner relationships. 


established mission organisations with 


On a more positive note, it cannot be overemphasised how critical the service providers 
and training institutes are for maintaining and improving the health of the populations 
they serve. In the DRC context, these non-state organisations are not only providing 
essential services but also helping to maintain a sense of normalcy in very challenging 
settings. The contributions made by the CFAs, while perhaps not making a large impact 
on their own, are part of supporting these vital services. The findings are provided below 
and in sections 3.2.1, 3.2.2 and 3.2.3 of DRC country report. 


Organisations that provide services: It was clear from interviews that those organisations 
that are involved in direct service provision are improving access to and use of health 
services in their areas of operation according to those providing and using them. The 
Diocesan Medical Offices (BDOMs) play a very important role in the DRC in health 
service provision, including managing certain health zones, running clinics and 
hospitals and providing community-based health services, as does the CME in Beni. The 
poor state of government health services in many of the zones where the various church 
services operate means that public medical care is not a viable option for health service 
users. 


BDOM Bukavu’s system of flat rate user fees (where all consultations and treatments are 
provided for the same fee, with no regard for how complex the health problem is) and 
support for health ‘mutuelles’ have had a particularly positive impact on increasing 

the poor’s access to church-based health services. BDOM Kananga and Kinshasa have 
reduced the cost of services for some specific interventions (malaria and caesarean 
section were mentioned specifically). However, beneficiaries across all three BDOMs 
indicated that user fees represent an obstacle to accessing certain services, especially 
hospital services. 


The BDOMs and CME Beni also assure a regular supply of medicines and other health- 
related material, to enable service users to receive appropriate treatment. These 
institutions have also, with their donors’ support, supported the construction and 
rehabilitation of health units. Concerns were raised in Kananga, though, as it would 
appear that some of the construction projects are left unfinished because the donor’s 
project period runs out before completion of the building. Government officials in 
Kananga were keen to see more commitment by Cordaid and other donors to seeing 
projects through to their logical conclusion. 


The Centre Olame in Bukavu has had a significant impact on the women helped through 
the centre’s counselling programme, as can seen from one woman’s comments ‘Olame’s 
activities for women who have been raped or sexually assaulted have helped to prevent 
deaths. Those who thought they could only find relief through suicide are little by little 
finding the will to live. The Centre Olame’s impact is limited by its small resource base, 
with demand for services far exceeding what staff can provide. 


3 
Mutuelles are local area, member-based insurance schemes where all members contribute an agreed 


ig money on a regular basis and the mutuelle then pays the member's user fees as and when 
ed, 
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The other service provider evaluated in DRC was the Goma-based Ecumenical 
Development Office (BOAD). Rather than providing health services, BOAD works through 
community-based organisations to provide emergency assistance to families affected by 
conflict. BOAD is changing from an emergency organisation to a development agency. 
Their interventions have been highly effective in supporting families in desperate need. 
Improvements on control by users over health services have been highly dependent on 
the management of the various health units and of the BDOM. While national policy 

is to have community health committees for every health unit, the degree to which 
these committees have been involved in planning and monitoring services varies. 

This is covered in more detail under ‘participation’ below. The impact of greater user 
involvement in managing health services is very positive where it has been supported. 
BDOM Bukavu and Kinshasa support to local health committees has helped to ensure 
not only active involvement of those committees in supervising health services but also 
has helped to ensure a greater participation of women in having a say over various 
community-based health activities, though women still remain under-represented on the 
committees themselves. 


The faith-based organisations covered in this evaluation tackled the serious issues of 
reproductive health and HIV/AIDS in highly variable ways. This ranged from an active 
HIV/AIDS prevention programme in Bukavu and Kinshasa to the non-existence ofa 
similar programme in Kananga. 


Impact and effectiveness could be improved in a number of ways: 


e Adopting more consistent pro-poor policies to ensure greater access to services by 
the poor. The various faith-based organisations are in a strong position to influence 
government policy on charges for services, to support other activities that would help 
to pay for the cost of care (e.g. local ‘mutuelles’ or insurance schemes), as well as to 
linkincome-generating activities to paying for care. 

e Establishing stronger links with government policies and priorities, such as priority 
national disease control programmes, e.g. the National AIDS Control programme and 
the National Malaria Control Programme. 

e Reinforcing the use of the National Health Management Information System to 
ensure a more standardised set of data is collected and can be compared within 
provinces and nationally. 


Organisations that operate as training institutions: Both health training institutions 
covered in this evaluation (IPASC and ESP) provide an invaluable contribution to 

health services in the DRC. Training is geared to the specific conditions of providing 
services in the DRC and African context, ensuring that graduates have appropriate 

skills for working within the DRC system. The ESP is the only institution in the country 
that trains public health professionals, and it was reported that these trainees are then 
employed in health zones and the Ministry of Health, in roles where their skills are most 
relevant. IPASC concentrates on training nursing and auxiliary health staff with public 
health understanding and skills in community-based health care. 


Organisations that advocate rights as well as provide services: These included CONAFED 
‘and ELIMU. The impact of these organisations would appear to be fairly localised, 
especially in the case of ELIMU. CONAFED has had some success at the national level: 
incease in number of women parliamentarians, inclusion of gender in the Family 
code, political negotiation meetings, getting gender more firmly on the agenda 


52 EVALUATION OF HEALTH THeme SEPTEMBER-DECEMBER 2004 


While the activities they support have been very important 


bers, etc. 
nv lagmeanaiaiees it f work has been limited by levels of staffing, 


for the communities they serve, their area O 
resources and the conflict. 


The evaluation team found that the management structures of these organisations . 
needed more institutional development support to be more effective. Strategic planning 
skills of staff are weak. More inclusive planning processes, whereby ELIMU in particular 
works with community constituents to develop its mission and strategies, would 

both improve communications with communities and strengthen the organisations 


themselves. 


5.3.2 Quality of services 
A number of indicators were used to consider the quality of services being provided. 


These included the amount of continuing professional development provided to 
professional staff, stock and drug supplies in clinics and validation of courses where 
training is concerned. Staff and service user opinions were also solicited. 


Organisations that provide health services: The evaluation found that the quality of 
services provided was generally good, despite the extremely challenging circumstances 
many of the services operated under. All service provider organisations have ongoing 
continuing professional development, through training and supervision, as part of 
their quality assurance system. Service quality is hampered by an overall lack of staff, 
limitations in infrastructure (especially in conflict-affected areas, e.g. CME clinics 

were looted and destroyed two years ago and current services are operating out of 
buildings not suited for the purpose.) BDOM Bukavu has introduced performance-related 
contracts and staff evaluations, which has had a positive impact on staff performance. 
Beneficiaries of BDOM services in Bukavu claimed that these were the highest-quality 
services available in South Kivu, due to the attitudes and professional quality of staff 
and the quality of the medicines prescribed. 


BOAD Goma has a monitoring and supervision system in place to ensure its partners’ 
activities are being carried out to agreed levels, though evaluators found that a more 
detailed strategic plan needed to be developed to help with measuring programme 
progress over time. Community facilitators within partner organisations are also trained 
by BOAD, helping to increase levels of participation from communities in supervising 
activities as well. 


Centre Olame’s quality was measured on the basis of the number of trained staff 
available to carry out the highly sensitive and specialist work of counselling traumatised 
women. Staff were found to be well qualified for their work, however evaluators felt 
that greater role differentiation between the different staff members could cut down on 
duplication of services and free up time to see a greater numbers of clients. Quality was 


also measured based on beneficiary perceptions of the services provided, which were 
very positive. 


Organisations that provide training: The training institutions evaluated adhere to 
national curriculum and international standards where they exist. Staff are provided 
with professional development. High standards have been assured to date at ESP due to 
rigorous entry requirements for students. 
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Organisations that advocate rights as well as provide services: Training modules run 
by CONAFED and ELIMU were developed based on the expressed needs of community 
groups and have been well evaluated in other assessments of the organisation’s work. 


5.3.3 Coherence 

Organisations that provide health services: All the service provider organisations 
evaluated work within the DRC health policy framework. The BDOMs manage a number 
of health zones in their provinces as well as run clinics. In the past the government had 
a laissez-faire approach to its partners and how they worked within individual health 
zones. The new government, under encouragement from the EC, is taking a more active 
interest in coordinating the activities of its partners, working through provincial health 
inspectors. 


There would appear to be some tensions between the state and faith-based organisations 
running health zones, due primarily to lack of clarity of roles and responsibilities, 
especially in relation to staff. For example, in Kananga, staff are unclear as to who is 
ultimately responsible for their contract (the state or BDOM), which is creating some 
problems. BDOM Kananga has also not implemented national priority programmes, such 
as the national HIV/AIDS strategy, though a programme is being put in place now for 
reproductive health. 


In general there is poor coherence across BDOM areas, as successful innovations 
developed by certain BDOMs are not being picked up by others. The Health Promotion 
Service of the Catholic Church would normally be the best body to help spread learning 
and innovation, but due to funding cuts‘ it has not provided the coordination service it 
was set up to provide. 


The BDOMs and CME Beni clearly link with Dutch CFA policies and priorities in 

making health services more accessible to communities that would otherwise not have 
services provided to them. Their community-based health programmes in particular 
improve access to primary care in its broadest sense, focusing as many do on water and 
sanitation as well as health promotion activities. Most of the work continues to be needs 
based, though rarely elaborated on the basis of systematic needs assessments within 
communities. Moves towards greater gender equity and gender programming in general 
are highly variable between the BDOM, demonstrating some lack of consistency with 
CFA gender policies. Interest in gender issues remains very dependent on those running 
the dioceses and has not been institutionalised across the BDOM in any consistent way. 


BOAD Goma conforms well with the DRC’s need for humanitarian assistance to those 
affected by conflict. They have become a key government and external donor partner in 
Goma for working to reduce the vulnerability of war-affected communities. 


In terms of coherence with CFA policies, there has been little development of rights- 
based approaches amongst these partner organisations. The evaluation team reflected 
that RBA may be difficult in the DRC context, especially in conflict-affected provinces, 
where the focus remains on basic survival for the majority of people. Even so, the 
BDOMs, CME and BOAD could work more through their own community-based 


4 Due to a difference in strategic view between the Health Promotion Service, which wished to move 
towards implementing projects and Cordaid, which felt it needed to develop its role as an umbrella 


organisation. 
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networks to support education programmes that support claiming rights, calling for 
more democratic structures and promoting peace, as well as supporting more demand- 
side initiatives that put communities more in control of their own health care (e.g. 


voucher schemes). 


The DRC government has no specific policy that targets the protection of women from 
violence and there are few programmes that support women who have been victims of 
violence. In this sense, the Centre Olame provides an example of important innovation 
that could and should be translated into a model for other programmes to follow. At 
present the Centre’s experience and learning remains highly localised, with staff under 
enormous pressure to respond to the day-to-day needs of their clients. Cordaid could 
help to support the sharing of Olame’s experiences to other like-minded groups in the 
DRC, and find ways to advocate for national level policy debates and strategies to 
protect and support women who are victims of violence. 


Organisations that provide training: As mentioned previously, both ESP and IPASC 
conform to national curriculum and standards. Their curriculum is highly appropriate 
to supporting the delivery of health care and public health in the DRC. Graduates are 
considered to be well prepared to work within the varied structures of the DRC health 
system. IPASC also operates a system of concessional rates for women who wish to 
study at IPASC, in order to redress gender imbalances in the health care workforce. 


Organisations that advocate rights and provide training: The absence of any national 
policy on the rights of women limits how coherent CONAFED and ELIMU’s advocacy 
work can be. As with the Centre Olame, these organisations are on the cutting edge of 
their fields. However, their ability to influence the development of policies and strategies 
remains limited, ELIMU because of its relatively small size and isolated position, and 
CONAFED because its secretariat is fairly small and it hasn’t yet identified strategies and 
specific targets for its work. The evaluators felt CONAFED in particular could put more 
effort into lobbying the NGO/CSO coordination body to promote a more coherent gender 
policy across civil society. 


5.3.4 Participation 

Organisations that provide services: BDOM, CME and BOAD are all active participants in 
government coordination meetings for the sectors that they are engaged in. Beneficiary 
involvement in the management of BDOM and CME services is much more variable. 
There is a national policy promoting the setting up of community health committees to 
support the management of health centres and hospitals, but the degree to which they 
are functional, and who is represented on these committees, is highly variable. 


BDOM Kinshasa has set up community health promotion groups, known as Mama 
and Papa Bongisa, made up of volunteers who define and promote health activities in 
their neighbourhoods. BDOM Bukavu actively supports community health committees 
to carry out planning and monitoring activities in their health centres, while also 
supporting the development of health mutuelles that help local communities raise the 


necessary funds to pay for health services. The role of BDOM Kananga in local health 
committees is unclear. 


CME Beni involves communities by working with local development committees, which 
provide health education and support user fees. However, there is poor participation 
of community members in managing CME-managed health centres, partly due to the 
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unstable situation in north-eastern DRC. Women’s participation in health committees 
and in the management structures of BDOM and CME remain deplorably low. 


BOAD’s approach is highly participatory, with local partners using participatory 
assessments to identify local needs. Local solidarity groups have been developed that 
have helped to create a greater sense of ownership of activities and of self-help within 
communities. The evaluators reflected that there is a real tension between delivering 
much-needed humanitarian assistance and increasing self-help initiatives within the 
communities where BOAD’s partners function. BOAD staff are keenly aware of this 
tension, but have little influence over international humanitarian assistance during crisis 
periods when aid pours into the region. BOAD plays an important role in the provincial 
mechanism coordinating humanitarian aid and has as such an import advocacy function 
for its partners. 


Centre Olame supports awareness-raising activities in communities through facilitators 
for community-level peace committees, though the evaluators found it difficult to assess 
how this relationship works in practice as there are no protocols for partnership or 
collaboration. Olame staff are themselves engaged in partnerships with other networks, 
such as regional Coalition Against Rape and Violence Against Women and the Human 
Rights Action Network. 


Organisations that provide training: ESP Kinshasa has a strong collaborative relationship 
with the government and sections of the University of Kinshasa. Former students work 
with ESP as resource persons to facilitate community level work for current students. At 
IPASC, students enrolling at the beginning of the year are asked to give input into the 
validation of the training programme as a way to encourage greater participation. IPASC 
has strong collaborative relations with other organisations in their area working on 
health and environmental issues. 


Organisations advocating rights and providing training: Members of the CONAFED 
network are involved in both planning and then implementing the network’s activities. 
More could be done to support these members to participate more actively and generally 
in civil society, as well as to advocate more strongly for women’s rights in government 
policy making. ELIMU, by the nature of its work, must work hand in hand with other 
organisations in its locality. Community groups that have received training from ELIMU 
have begun organising themselves into local committees that are founded on more 
democratic and egalitarian principles. 


5.3.5 Sustainability 

For the foreseeable future, long-term financial sustainability of all partner activities 

in the DRC will remain dependent on external resources. User fees can only provide a 
small percentage of the’ operating costs of health services, and of training institutions, 
and in themselves are a substantial barrier to the poor. However, other aspects of partner 
activities have led to sustainable improvements in the communities where they work, 
leading to greater sustainability of programme outcomes, institutional mechanisms and 


human capacity. 


Organisations that provide services: Where partners such as the BDOMs give substantial 
support to local health promotion groups and health management committees there is 
clearly greater community ownership of certain health activities. In Bukavu, BDOM's 
strong connections at community level have helped to ensure the sustainability of 
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ons and looting due to the conflict. Another apparent 
feature of sustainability is seen in those organisations with strong governance on 
structures, such as CME Beni. The board of governors is strongly involved in deciding 
what new areas of action the CME should move into, basing part of its decision on the 
and financial burdens these new areas Impose. 


services despite frequent disrupti 


management 


BDOM Bukavu has moved towards greater financial sustainability by manufacturing 
essential drugs and packaging, though there are concerns that the health service 
provision part of the BDOM’s ‘business’ could represent too much of a financial burden 


on the manufacturing side of the business. 


At present the Centre Olame’s future sustainability is questionable. It is not only heavily 
dependent on external funding (which is appropriate given its clientele), but it has also 
failed so far to establish linkages into local communities and with local government. 
Given the invaluable service provided by the staff at the Centre its donors should 

be working with centre staff to develop long-term plans, in association with local 
stakeholders, that will help the Centre’s work brave any possible future disruptions to 
funding and secure longer-term support. 


Organisations that provide training: Both ESP and IPASC will remain financially 
dependent on external funding for some time to come. Their strengths lie in the fact 
that, due to the quality of the training and other services they provide, they are able 

to acquire reasonable amounts of funding. The substantial involvement of former 
students in the community training programmes of ESP allows further opportunities 
for sustainability, as this has created a community of public health trainers that can be 
drawn upon. 


Organisations that advocate for human rights: Both CONAFED and ELIMU support 
groups and activities that work towards sustainable improvement in women’s status 
overall. While their area of activity is admittedly limited, the individuals who have 
benefited from literacy training, income-generating activities and involvement in 
community structures have developed lasting skills. Both organisations need further 
support to scale up the work they do to reach larger numbers of people and achieve 
greater impact. 


5.3.6 Added value 

Organisations that provide services: The added value of the relationship with the CFAs 
for BDOM and CME is not only the flexible funding made available to them by Cordaid 
and ICCO. These partners also felt that CFA funding had helped them to leverage 

other external funding, and, particularly in the eastern provinces, has allowed these 
institutions to respond to health-related problems arising from the conflict in ways that 
they otherwise couldn’t have managed. In the case of CME and BOAD, both partners felt 
that ICCO’s support had helped them to develop much greater organisational capacity 
to respond to the humanitarian crises generated by the conflict in their areas. Another 
observed significant added value of ICCO’s support is that it enabled partners to adopt 
and maintain a more holistic approach to health, which doesn’t address health in 
isolation but as an integral aspect of development. 


Centre Olame has benefited from being linked to wider networks that are dealing with 


similar women’s rights issues to those addressed by the Centre’s work as a direct result 
of its partnership with Cordaid. 
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Cordaid and ICCO risk having a fragmented programme in the DRC, which reduces the 
potential added value they can provide. While Cordaid has a strong ‘common thread’ 
running through the support it gives to the BDOMs, this isn’t as effective as it could 
be without there being an organisation providing the umbrella functions formerly 
carried out by the SPS. It could be said that Cordaid is providing some of the umbrella 
institution functions, at least for Catholic health services, that are being provided by 
national Church Health Associations in other countries. However, Cordaid cannot and 
should not fully play this role. Its other partners have little connection with each other, 
except through meetings facilitated by Cordaid, and the evaluation team observed that 
innovative practice and lesson learning isn’t being emphasised enough to influence 
changes in more recalcitrant partners (e.g. transferring experience gained from BDOM 
Bukavu to BDOM Kananga). 


ICCO’s programme is determined primarily by the strategies and activities of its partners 
with no clear connection or learning between them. While this provides maximum 
flexibility to partners it does lessen the impact that ICCO’s support can have on its 
priority policy concerns. The lack of a country strategic framework also means that ICCO 
programme officers have little guidance to go by when assessing whether to fund new 
organisations or new activities, and there is no coherent way to link the achievements of 
partners in the DRC to ICCO’s global basic services strategy. 


Neither CFA uses its influence to develop a more gendered approach to the work of their 
partners. This is particularly apparent in Cordaid’s relationships with the BDOMs, which, 
as can be seen above, have highly varied responses to both gender concerns and sexual 
and reproductive health. Given the conservative and patriarchal tendencies of many 

of the Catholic and Protestant churches and missions, both ICCO and Cordaid need to 
decide whether continued support to organisations that will not heed calls for greater 
participation of women in decision making, both at community and organisational 
levels, constitutes colluding with this more conservative agenda. The evaluation 
findings indicate that traditional approaches are not being challenged sufficiently and 
that Cordaid in particular needs to exert more progressive influence over its Catholic 
partners. 
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Findings: Nepal 


Country context 


Nepal has been in the throes of violent conflict in parts of the country for the last eight 
years, with more than 10,000 civilian lives lost so far. This insurgency has made the 
running of development activities very difficult in the remote and rural parts of the 
country, which are also the most deprived and needy areas. Development infrastructure 
has been badly damaged, while security forces and the insurgents have committed 
human rights violations with apparent impunity. Many NGOs and other development 
agencies are facing threats and pressures from both sides in the conflict, and some have 
been forced to stop their activities altogether in Maoist-controlled areas by the Maoists 
themselves. 


Nepal, like many countries, has been undergoing political and structural reform, 
following a policy of decentralisation from national to local government. Services such 
as primary health care, education, postal services and agricultural extension work now 
report to local authorities, though the political disturbances have badly affected the 
effective running of these authorities. 


Health services are multi-tiered and very clinic-focused in Nepal. While the government 
continues to provide the majority of health services for the Nepalese population, private 
service providers and NGOs are also major players in the health system. The government 
also promotes Ayurvedic medicine as an important alternative to western-style 
medicine. Donor coordination in most sectors has been poor, though there have been 
recent moves towards greater donor harmonisation. Special development interventions 
for health, education, and local government are to be launched for neglected areas of 
the country. The new Ministry of Health and Donor Health Forum will tackle health 
sector development and public sector reform. The health sector does not currently have 
a meaningful decentralisation strategy and practical devolution is a critical reform 
issue. Also there is a plan to explore more formal contractual arrangements between 
the government and non-state providers of services. More on the Nepal context can be 
found in Nepal country report, section 2. 


Table 8 provides some key health indicators for Nepal. 


Table 8 - Key health data for Nepal* 


Infant Mortality Rate 66/1000 
Under Five Child Mortality Rate 91/1000 
Maternal Mortality Rate 740/100,000 
Life expectancy 59.9 years 
Total Fertility Rate 4.3 
HIV/AIDS prevalence 0.3% 
. <5 underweight for age 31% 
<5 fully immunised against TB 85% 
<5 fully immunised against measles 71% 


* from UNDP (2004) Human Development Report 2004 
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6.2 CFA policies and programmes in Nepal 


Table 9 - Country-specific health policies 


Plan Nepal 


Improved health and nutritional status of 
children and families (working primarily 
through Early Childhood Care and 
Development Centres, ECCDs). 


ECCDs have been written into the 
Nepal Poverty Reduction Strategy, 
with government hoping to set up and 
maintain 74,000 ECCDs nationwide by 
2015. 
e Fears that minimum standards 
for ECCD have not been adequately 
considered - nutrition component is 
missing from government plans. 


No specific country health policy 


licies/Objectives 
ela ICCO does have a Nepal country strategy. 


Conflict situation has led to the suspension of most local 
government service delivery. ICCO partners work closely 
with communities, are not being targeted by either 
government or insurgents, so policy of demand-driven 
development appears appropriate in the Nepal context. 


Fit with Nepal 
government policy 
and strategy 


The Dutch Government no longer has a bilateral programme with the Government of Nepal, though 
a few projects are being supported to their conclusion. Therefore Dutch aid for Nepal is put through 
NGOs such as ICCO, Plan and SNV. 


Fit with Dutch - 
Nepal government 
policy and 
strategy 
Partners and 
health-related 
funding 


Plan Nepal focuses on improving the 
health and nutritional status of children 
and families working primarily through 
Early Childhood Care and Development 
Centres (ECCD). Main health-related 
activities include immunisation, 
diarrhoeal disease control, acute 
respiratory disease control, nutrition, 
reproductive health, water and 
sanitation, as well as rehabilitation of 
physical infrastructure. 


UMN-YUHP: Provides basic health care services to carpet 
industrial workers living in poor neighbourhoods of 
Kathmandu. 


CCDN: Newly created NGO (from a UMN project) providing 
services to ethnic minority communities in Makwanpur 
District. 


CBRS: Supports physically disabled people and their families 
in Pokhara District, through community-based activities 
and home visits. 


SSS: Kathmandu-based organisation supporting HIV and 
AIDS activities, primarily in Information, Education and 
Communication. 


BNMT: International NGO providing direct health services in 
Nepal in a number of districts. 


CVICT: Provides support to victims of torture and training 
for NGO workers in counseling. 


WOREC: Community-based, women focused organisation 
supporting reproductive health and income-generating 
activities, as well as campaigning on trafficking of women. 


Plan Netherlands support to Nepal 2004 
€ 1,486,647, of which € 139,151 for 
health 


Total health-related expenditure = € 338,954 in 2004 
Total expenditure in Nepal was € 1,333,563 
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6.3 Results 


epal evaluation were found to be service providers 


as far as their health-related activities are concerned, except for SSS, which to date has 
provided information, education and communication support on HIV and AIDS to the 
Nepal AIDS Programme, along with some outreach to people with HIV/AIDS. Some of 
the organisations function as umbrella organisations operating through community 
organisations (e.g. WOREC), while United Missions of Nepal (UMN), which founded some 
of the organisations covered, still links the organisations they have spawned. WOREC 
and CVICT also have strong human rights advocacy elements to their programmes. 

As all the organisations involved in this evaluation were primarily service delivery 
organisations, or the components of the evaluation were service delivery focused, there 
is no differentiation in the reporting of findings between service delivery and network 
organisations in this chapter. More detailed information on findings can be found in 


separate country report, section 4. 


All organisations included in the N 


6.3.1 Impact/effectiveness 

CFA partners themselves have few, if any, strategic objectives and SMART indicators 
against which quantitative impact or effectiveness could be measured. However, due 

to the conflict in Nepal, as with DRC, many of the CFA partners are the only health 
care providers for whole sub-districts or even regions (in the case of BNMT). Their 
withdrawal from these areas would mean no health services would be available for the 
communities currently being served. The figures given aboveconsisting of 2003 or 2004 
data provide a view of the average annual health activity level of the organisations 
evaluated, as these are very similar to service use levels from previous years. Due to the 
conflict situation in Nepal government health services barely function at all, making 
non-state health services that much more critical to communities. 


ICCO’s lack of a country strategy framework makes it impossible to measure progress 
towards either country or global policy objectives. Furthermore, lack of baseline data 
for most of the health indicators amongst Nepalese partners made it very difficult to 
make a comparative analysis of quantitative results of the work of partners. As with the 
DRC programmes, there is no discernable association between length of partnership or 
commitment of resources to the partners by ICCO, as most of the longer-term partners 
are recipients of multiple sources of funds. Plan, on the other hand, is able to provide 
concrete evidence of progress against programme objectives by having measurable 
indicators and regular progress reports that monitor progress against these indicators. 


The evaluation team did try to compare HIV/AIDS awareness and DOTS (Directly 
Observed Treatment of TB) using data of some partners and national data, notably 
for BNMT and Plan, but found that the poor quality of the national data rendered the 
comparison meaningless. 


The data below compares the proportion of women giving birth at home in the workin g 
area of BNMT, Plan and YUHP with the national figure. Deliveries at home are less than 
the national average in the programmes of all three partners. However, in YUHP area 
and in much of the operating area of BNMT women do have access to hospitals and 


5 dBase . 2 H 

Safe Motherhood initiatives emphasise the importance of having skilled birth assistance and ready access 

f “mergency obstetric care during labour and birth to avert maternal illness and death. Home births tend to 
€ attended by unskilled family members. 
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health centres providing delivery services, so the figures for home delivery could be 
considered as very high. Whereas use of the safe delivery kits® during delivery is very 
low at national level as well as in BNMT area, their use in the Plan area is very high. No 
data on the use of safe delivery kits for deliveries at home were found in YUHP area. 


Deliveries at home % Use of Safe Delivery Kits % 
National 89 National 9.4 
BNMT 80 BNMT 30 
PLAN 77 PLAN 77 
YUHP 70 


As set out in Table 10, most of the other partners have specific targets set out in their 
project proposals and the trend for achieving the target is on the positive side. The 
main issue here is that most of the partners do not have any baseline data related 

to their interventions. Most of the available data are input/output data only, which 
demonstrates the effective delivery of the project and its target achievement but makes 
it difficult to attribute overall impact. For example, CVICT’s targets largely depend upon 
the needs of the torture victims it caters for, and are based on the reported cases only. 
CBRS has a target of serving 825 physically disabled persons during the current project 
period, which it is likely to achieve. But it has no baseline data of how many physically 
disabled people there are within their operating districts or Village Development 
Committees (VDCs) 


Qualitative information on impact is generally very positive. National government views 
on impact (see Nepal Country Report, section 4.2.) were confined to those projects that 
Ministry of Health staff were aware of, which were the BNMT and Plan programmes. 
MOH officials expressed high appreciation for the contribution made by BNMT to 

the DOTS programme in Nepal’s eastern region. The view is that BNMT’s efforts have 
meant that the poor in this region have benefited in particular, as they are also the most 
affected population. 


Plan is recognised by the national and local government as a major contributor to 
health- and education-related programmes in the areas where they work. Plan Nepal 

is considered as a long-term supporter, pioneer and champion of running the Early 
Childhood Care and Development Centres (ECCD). The impact of Plan’s programmes is 
considered to be mainly in strengthening the delivery capacity of government health 
workers and volunteers, improving service quality and improving accessibility to basic 
health and nutrition services of mothers and children. Plan’s ECCD programme and 
advocacy work have led to the inclusion of ECCDs in the Nepalese Government’s PRSP, 
which has set a target of putting 74,000 ECCDs in place by 2015, though Plan could do 
more to ensure that the government strategy adheres to a higher standard programme. 
Currently, for example, the government is proposing to not provide any food for 
children in government-run ECCDs, even though this is a core element of the Plan ECCD 
programme, and it is vital to improving the poor nutritional levels of Nepali children. 


Kathmandu municipal authorities are very appreciative of YUHP’s work in improving 
health service accessibility and availability to the urban poor and migrant population 
living in one marginalised area of Kathmandu. 


6 The percentage of births using safe delivery kits gives a good proxy for whether women have been assisted 
by someone with some formal training. 
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sipabesraennante! h team were generally satisfied with the quality 


Beneficiaries interviewed by the researc 7 
of services being provided. The government line agency staff also expressed their 
NGOs. Skilled and qualified personnel 


, ; ne 
satisfaction with the quality of the services of | : 
(with required technical competencies) are engaged in the delivery of the programmes by 


all partners, and this is viewed as an indicator of service quality by those interviewed. 


Regular access to drugs and supplies were another indicator of quality for health service 


provision. 


6.3.3 Coherence C 
Partner programmes are both coherent with government and CFA policies, where they 


exist, and, in some cases, are at the cutting edge of policy, helping to influence the 
creation of new pro-poor policy. The key areas of convergence include: 


e In line with ‘Health For All’ and ‘Long Term Health Plan’ of the government (but not 
strategically planned by small NGOs) 

e In line with the government’s priority for primary health services 

e Promote ‘Right-Based Approach’ for health and nutrition (in the PRSP) in service 
delivery but not in advocacy strategies 

e Indirect contribution to the livelihood of the beneficiaries (e.g. through ECCD’s 
development activities, DOTS TB control and RH/MCH) 

e Pro-poor focus and involvement of community and local agencies are pursued by the 
programmes 

e Many programmes support strategies for assuring gender equity and women’s 
empowerment (ECCD, CVICT, SSS). 


However, poor-quality baseline and quantitative data create difficulties in attribution 

of the target achievement in macro-level indicators. In other words, it is difficult to 
measure the degree to which the partners’ programmes are contributing to overall Nepal 
MDG and PRSP targets. 


YUHP has worked on filling a policy gap through its innovative work with the urban 
poor. Nepal has no urban health policy and the health system has not yet created 
structures to cater for urban populations. YUHP’s approach of involving the municipal 
authorities from the beginning of the project has led to the municipality first increasing 
the levels of staffing and funding for a municipal health office, and now being in a 
position to have YUHP clinics handed over to municipal control. This project should 
serve as a model for other urban health service provision. 


It is unclear the degree to which United Missions Nepal has worked with the government 
on incorporating some of the key systems concepts introduced by this project, which 
represents a missed opportunity for influencing government policy in this area. 


Plan’s projects emphasise capacity building not only of local community organisations 
but also of local government staff. Their capacity-building work is very well regarded 
by local and national government, and has probably played a useful role in ensuring 
that Plan’s approach to early childhood care has become institutionalised within the 
government’s medium- and long-term plans. 


In some cases local authorities reflected that they would wish to have closer contact 
with some of the organisations, and a better understanding of their activities. However, 
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the evaluation team also found that these same local government staff realised that the 
current conflict probably necessitates organisations keeping a certain distance from 
government structures. This understanding reflects both the balancing act performed 
by organisations working in Nepal today (especially in rural areas) and the maturity of 
local government officials. 


CVICT works with an extremely vulnerable population, and somewhat outside 
government policy frameworks. CVICT’s work is primarily supply led at present, 
providing individual counseling and support to people who have been tortured. 


Given the scale of the problem in Nepal (it is estimated that 200,000 women and 
children are ‘missing’ from their homes due to trafficking) CVICT and WOREC could do 
more in advocacy to encourage government decision makers to respect and promote 
legislation on protecting and promoting the rights of women and children and to 
enforce existing legislation on trafficking. 


6.3.4 Participation 

All service provider projects adhere to principles of community participation and 
management of health services to a great extent. Community management committees, 
women’s groups and community-based organisations are being promoted by service 
provider partners, where communities are encouraged to play a role in managing health 
programme activities. This includes planning of field activities with community groups 
and providing training to community-based organisations. The social structure of Nepal 
(caste system and low status of women) has also meant that most programmes place an 
emphasis on ensuring the inclusion of marginalised groups, including those of a low 
caste, women and, in the case of Plan, children. 


BNMT, the various UMN associated programmes and Plan put a great deal of emphasis 
on promoting community participation and ownership of their activities. One key 
indicator of the success of the participatory approach of all these organisations is that 
the Maoist insurgents, having assessed how the projects operate, have decided that they 
meet community priority needs and the insurgents thus largely leave the staff alone to 
get on with their work. CCDN was highlighted by the evaluation team as managing to 
work in especially difficult circumstances by maintaining a good relationship with line 
agencies in local government while also being seen to be addressing the key needs of 
the poor and vulnerable in their project area. 


Plan’s work at community level includes employing community facilitators, who in turn 
work closely with community volunteers who contribute enormous amounts of time 

to project activities. Women volunteers in the ECCDs take turns to look after children 
and provide food for the one meal served to children at the centres. The evaluation 
team found that more support needs to be provided to some Plan staff to help them 
overcome some of the prevailing social attitudes. For example, in one area visited, staff 
were asked how the rotation system for food preparation at ECCDs worked in practice 
where members of the rotation included Dalit women. The staff replied that, in this 
case, other non-Dalit women in the rotation list prepared the food rather than allowing 
the Dalit mothers to do so. While acknowledging that changing social sensibilities 

is very difficult, the team felt that because Plan has built up such good relationships 
with its programme communities its programmes could be doing more to challenge 
those attitudes and beliefs that continue to exclude Dalit women from many activities. 
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However, community-based staff will need training and support in order to facilitate 


this. 


With regards to the participation of young people in Plan’s propraimee as the 
programmes assessed for this evaluation considered only the early years centres that | 
cater to children under 5 and their mothers, the research team focused on the mothers 


participation and control over health-related activities. 


6.3.5 Sustainability sgaitioes 
As with the programmes in the DRC, the financial sustainability of development 


organisations in Nepal is weak. Most programmes will remain dependent on external 
funding for the foreseeable future, as even if the conflict should be resolved, it will take 
a long period of reconstruction and economic growth to assure that both the state and 
non-state service providers have some ability to be financially viable. 


The sustainability of the organisations themselves and of the changes they have 
promoted in the communities they work in is more variable. CBRS was highlighted by 
the evaluation team as having a particularly strong governance structure, where board 
members were both independent of implementing staff and also important individuals 
with high local profile who are committed to assuring the ongoing work of CBRS. This 
type of governance structure lays the foundation for supporting the maintenance and 
expansion of CBRS’s work in future, though there are other constrains on sustainability 
(see below). In contrast, CCDN’s governance structure is staff driven, so that the 
organisation could find itself in the somewhat compromised position of having its 
accountant also being the treasurer (there are no organisational rules that would prevent 
this at present). 


The strong emphasis on community management and ownership of programme 
activities in all organisations, except CBRS and CVICT, also bodes well for the future 
sustainability of programme activities. The evaluation team felt, for example, that 
Plan-supported community structures had very high levels of voluntary participation 
and engagement, which could mean that basic aspects of programme activities could 
be sustained even without ongoing Plan funding. CBRS and CVICT, despite their very 
special service area and very effective service delivery, have perhaps the weakest 
sustainability situation. The phase in and phase out strategy, service content and 
spectrum and community-based approach are the areas where these organisations need 
to put more effort to ensure sustainability of their services after CVICT and CBRS phase- 
out their support. 


The sustainability of SSS was raised as a concern by the evaluation team, as it appears 
to have not found a particular niche for continuing its HIV and AIDS work, nor does it 
have a particular constituency that owns and supports its work. This is especially true 
now that more actors have become involved in supporting HIV and AIDS work in Nepal, 
so that competition for funding in this area is growing stronger. 


BNMT’s work is highly integrated into government systems, and in fact works as 

a substitute for government structures in the Eastern Region. While the level of 
integration should be an indicator of longer-term sustainability of programme 
activities, there are concerns that, should BNMT support withdraw from the region, the 
government would not have the resources to continue the work. The evaluation team 
strongly felt that BNMT needs to put more emphasis on the longer term, by identifying 
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handover arrangements with government authorities. In contrast to this, YUHP provides 
a model of programme sustainability, where programme managers built sustainability 
into their programme from the outset. This was done through active engagement with 
the municipal authorities and devising a plan to handover project activities to the 
municipality after a certain period of UMN support. 


6.3.6 Value added 
The partners in Nepal were universal in their acknowledgement of the many positive 
aspects of working with ICCO and Plan Netherlands. These aspects included: 


e The CFAs provide stable resource support that allows the Nepal partners to make 
longer-term plans. 

e The flexibility in budgeting and use of funds enables the partners in Nepal to respond 
to the needs of the community in an ongoing manner. 

e The CFAs are responsive to the needs of their partners in Nepal, and do not interfere 
much with implementation. In essence, they are left to get on with their work. 

e In the case of SSS, their relationship with ICCO has allowed them to be involved in 
regional and international HIV and AIDS networks, which staff have found highly 
beneficial. (Although it is unclear how this has translated into taking a more strategic 
approach to their work). 


However, ICCO’s country strategy for Nepal is less defined than for the DRC, and there 
is no apparent cohesiveness in terms of who is supported, other than the fact that 

each partner does share similar values to ICCO, and their activities fit within ICCO’s 
priority policy areas. This does limit the added value that ICCO can bring to its Nepalese 
partners. 


Also, limited direct monitoring and changes in Netherlands-based staff (in particular 
ICCO) creates confusion sometimes, with partners unclear about how shifts in policy 
direction have come about and how they should respond. There appears to be little 
support provided by the CFAs for the organisational development of their Nepal 
partners. Partners indicated that staff turnover at ICCO inhibited developing a longer- 
term working relationship with the organisation. Also, each ICCO programme officer 
covers a number of organisations and projects. This means that they have very little 
time to work in depth with any individual project and cannot, therefore, provide 
institutional development support. Partner representatives also commented that the 
workshops run by this evaluation were the first occasion ever where they had met as 
CFA partners and that they would welcome more opportunities to do so. 


In terms of partnerships, the organisations interviewed felt that their key partnership 
was both with the communities that they worked with and with government. They 
emphasised the linkages and coordination with the Nepal Government’s line agencies in 
the areas they worked in. However, the vision and commitment of some of the partners, 
and in particular field-based staff, to institutionalising their work along the lines of 
government policies in the longer term was very limited. 
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7.1 


Findings: Zambia 


Country context 


Zambia is a landlocked Southern African country with a population of approximately 

10 million people. It is considered to have one of the most urbanised populations in 
Africa, a feature that is due primarily to Zambia’s mineral extraction industry, which 
has led to the creation of various urban centres. Since independence in the 1960s 
Zambia has unfortunately experienced a large decline in social and economic indicators, 
caused primarily by a dramatic fall in the world market for copper, one of Zambia's 
primary industries. It is now one of the world’s least developed countries though, unlike 
DRC and Nepal, benefits from the fact that it has a stable and increasingly democratic 
government, and no internal conflict to contend with. 


The Zambian health system has undergone public sector reform, including a robust 
decentralisation process. Health services are managed by District Health Management 
Teams now, rather than through the national Ministry of Health. The donor community 
in Zambia has begun to coordinate its assistance more in order to provide better support 
to the government’s poverty reduction strategy. In the health sector, a number of donors, 
including the Dutch government, now pool their assistance in a basket fund for use 

by the Ministry of Health. Negotiations are underway to move towards a Sector Wide 
Approach (SWAp) in health in the near future. Zambian health infrastructure is poor and 
levels of staffing are very low compared to what they should be in both government 
and non-government health services. The situation has become more acute as the most 
qualified staff seek employment outside Zambia, either in the region or in northern 
countries, in an effort to improve their families’ economic position. Another part of the 
health sector reform process was the introduction of user fees for services across the 
country. 


About 40% of health care in Zambia is provided through the non-state sector, with 
faith-based organisations making up the largest proportion of non-state provision. The 
government has increased its share of funding to its non-state partners from 25% to 
75% of church facility running costs to cover loss of other income for especially mission 
hospitals. 


A recent Ministry of Health report (2004), which reviews the progress of the health 
sector reform, has found that in general districts have taken on planning activities, 
though district teams néed to do more to prioritise what they will do. It was also noted 
that the current district planning format gives no space to explore cross-cutting issues 
such as mainstreaming of gender or poverty issues or ways to improve equity. The MOH 
review also notes that the impact of user fees has been fairly negative, as the revenue 
generated is tiny compared to what is needed, and that fees have had a deterrent effect 
on service use by poor people. 


The Zambian population, as with all of Southern Africa, has been heavily hit by the HIV 
and AIDS pandemic. This has led to severe social and economic problems, as well as the 
individual health problems for those diagnosed as HIV positive. One of the most striking 
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impacts of the level of HIV infection in the country is the fall in life expectancy a 
32.7 years. HIV and AIDS have also had a severely detrimental impact on key public 
services such as health, as many workers have become ill and died from AIDS. Key 
health indicators for Zambia are in Table 11. Further information on the Zambia context 


is available in Zambia country report, section 1. 


Table 11 - Key health data for Zambia” 


Infant Mortality Rate 108/1000 
Under Five Child Mortality Rate 192/1000 
Maternal Mortality Rate 750/100,000 
Life expectancy 32.7 years 
Total Fertility Rate 5.6 
HIV/AIDS prevalence (%15-49) 16.5% 
<5 underweight for age 28% © 
<5 fully immunised against TB 92% 


* from UNDP (2004) Human Development Report 2004 


The Dutch government provides significant support to the health sector in Zambia. It 

is embarking on a pilot programme with the government that aims to motivate more 
qualified staff to move to and stay in health centres and hospitals in more remote areas 
of the country. The programme is introducing a package of incentives to health staff, 
which includes subsidised salary payments, improvements in living accommodation and 
working environment, as well as school fees for school-aged children. 
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7.2 CFA policies and programmes 


Table 12 - Country-specific health policies and programmes 


| | Corda 


Policies/ Contribute to improving the quality of and access 

Objectives to health services (preventive and curative) for 

the poor and vulnerable. Specific objectives of 

CORDAID's health programme are: 

¢ Human resource development 

e Capacity building of organisational and 
institutional development 

e HIV/AIDS 

e Basic health services and community-based care 


Improve the quality of health among children and 
mothers in Plan communities: 

* Increased access to basic primary health care 
that provides integrated health services based 
On cost sharing 

Reduced disease levels by increasing awareness 
of PHC, watsan and nutrition 

¢ Greater emphasis on HIV/AIDS interventions 

* Increased access to potable water 

Community management to maximise cost/ 
benefits and utilisation of volunteers linked to 
continued capacity building 


Fit with Zambia 
government 
policy and 
strategy 


Strong decentralisation and integrated service 
funding and delivery programme. CORDAID 
partners are mostly church-based service delivery 
organisations, are also key partners in the 
government's overall service delivery strategy. 
Funding to support partners working in advocacy 
for human rights, monitoring PRSP. 


Plan works closely with the Zambian government 
in building infrastructure and training district 
health staff. 


Fit with Zambia | Dutch government health programmes in Zambia are focused on human resource development, in 


- Dutch particular training and retention of health staff. 10 year commitment to providing increased incentives to 
government staff to work in more remote areas and to remain within government service. 
policy 


Plan Zambia: Main areas of action in Zambia 
include the provision of integrated basic primary 
health care services, water and sanitation 
activities, health and nutrition promotion, HIV 
and AIDS prevention activities and community 
capacity building through community facilitators. 


Partners and 
health-related 
funding 


CSPR: A national network organisation supporting 
provincial and district poverty monitoring 
groups who monitor government progress on the 
National Poverty Reduction Strategy 

CCJDP: Supports parish-based organisations with 
training and mobilisation activities in the areas 
of poverty reduction, social action, HIV and AIDS 
and gender equity. 

CHAZ: Faith-based community’s umbrella 
organisation made up of 90 faith-based 
institutions. Liaises with the Zambian Ministry of 
Health on behalf of its members 

Dioceses: Run hospitals and health clinics 
throughout Zambia, as well as community-based 
health programmes. Ndola Diocese manages an 
integrated AIDS programme 


Programmes funded at 100% but exact figures 
were not provided. 

Health activities in the 3 regional programme 
units in Zambia, operating in Mazabuka, Chadiza 
and Chibombo districts of Southern, Eastern and 
Central Provinces 


Total Plan Netherlands Zambia budget: 


€ 657,356 
Overall health budget 2004 - € 400,116 


Total Cordaid budget 2004: € 2,328,829 
Total committed to health 2004: € 498,825 
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Finpincs: ZAMBIA 


Results 


The above tables indicate that there is a considerab] 
supported by CFA partners in Zambia. Many of the 
support for several years, with the CFA providing a 
funding their partner organisations require. As with 
not have health-related indicators related to their w 
have very specific health-related impact indicators 
against indicators provided in their annual reportin 


€ amount of health-related activity 
programmes have received CFA 
variable percentage of the overall 
the DRC, the Cordaid partners do 
ork. On the other hand Plan does 
and detailed mapping of outputs 

g system. 


As with the other countries, the faith-based network supported by Cordaid and the 
operational programme of Plan are making an important contribution to health 
service provision in Zambia, as reported both by community members interviewed and 
government counterpart organisations. More detailed results are provided below. 


7.3.1 Impact/effectiveness 

Organisations that provide services: The majority of organisations included in this 
evaluation have significant elements of health-related service provision in their 
programmes. These include the Dioceses of Ndola, Mpika and Mansa supported through 
Cordaid and Plan Zambia’s programmes. 


79:2 Quantitative 

As with the DRC Cordaid partners it is difficult to measure quantitative results of the 
dioceses and advocacy programmes supported by Cordaid. The evaluation found that 
health indicators for CFA-supported units show little difference to rates found in the 
national or district health information system (see Zambia country report, section 3.2 
for detailed health information). As the programmes themselves do not have baseline 
data it is impossible to build a ‘before and after’ picture of the quantitative impact of 
health services in any particular communities. In some cases the length of operations 
of the diocesan health services would make baseline data meaningless, so that the only 
comparison is with comparable districts and national level statistics. However, the 
indicators used within the health system and within specific institutions have changed 
over time, also making it difficult to do any trend analysis. 


With the above caveats in mind, it is again useful to consider what the health situation 
would be if Cordaid’s partners were not providing or reducing the provision of health 
services. As faith-based organisations make up almost 40% of Zambia's health care 
system, they play a critical role. Cordaid’s partners also go beyond providing just 
curative health services, by supporting an extensive range of community-based health 
care. In particular, Ndola Diocese’s AIDS outreach programme is highly celebrated and 
seen as a model to be replicated in other countries. 


Plan is making steady progress against this current period’s indicators, with annual 
targets more or less being met. Respondents in Plan project areas also reported that 
there had been a noticeable decline in water-borne infections, especially diarrhoea, as a 
result of Plan’s water and sanitation programmes. 


7233 Qualitative al 
Government stakeholders that were interviewed felt that the health projects supporte 


by Cordaid and Plan partners were making a meaningful impact on thelr aca aa ‘ 
catchment areas. In terms of health service delivery, access to services Is assured al bo 
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s and hospitals) and within communities. In Nchelenge 
ported hospitals of St. Paul’s and Chilonga Mission 

so they serve a larger population area than 

t also runs a district hospital, the 


institutional level (health centre 
and Mpika, the two Cordaid-sup 
also serve as first-level referral hospitals, 
just their districts. In districts where the governmen 
Cordaid-funded hospitals tend to have more qualified staff. 


Plan was also commended by government stakeholders for supporting the building of 
seven health centres in Mazabuka (out of a total of 42 in the district). This was seen as 
an important support to the government's infrastructure as the DHMT couldn’t afford to 


build new centres from their own funds. 


District teams also appreciated the community level work offered by the different 
CFA partners. Besides the support to infrastructure it was felt that the importance 
given to community capacity building was very valuable. Plan was singled out for its 
work in community mobilisation, whereby user groups are set up and trained by Plan 
community development facilitators. As this work is undertaken in collaboration with 
local government staff, district staff felt there was a high degree of synergy between 
Plan’s operations and their own programmes. 


Organisational staff also felt that programme impact could be measured through support 
to health service infrastructure and clinical activities, as well as through support given 
to community-based activities. As a result of working at both levels (health structures 
and community) access to and use of services had improved greatly. However, staff also 
felt that user fees were a significant deterrent to poor people’s access to services, and 

in Mpika District, declines in service use were directly attributed to the introduction of 
a by-pass fee. At Our Lady’s Mission hospital, for example, the poor financial position 
of the hospital and high running costs (partly resulting from inadequate and erratic 
funding from government) meant that the beds in the hospital were almost all empty 

as the hospital’s management was very strict in requiring that individuals pay their 
user fees before using services. Even though the government has put in place a user fee 
exemption policy, this is generally too difficult to implement and most poor people do 
not attempt to use health services for fear of being charged anyway. 


In all programmes visited, it emerged from staff interviews that there was a strong 
presence of, and support for, community-based health and health-related services. These 
are felt to be making a profound impact on the quality of community well being. For 
example, home-based care programmes and training of home-based care providers for 
HIV and AIDS are taking place in all sites visited by the evaluators. It was felt that the 
effects of the home-based care programmes included: 


¢ An increased number of women delivering babies under the supervision of a trained 
attendant 

Better access to treatment by community health workers 

Better nutrition levels and availability of growth-monitoring services for under 5’s 
Greater HIV and AIDS awareness in the community on prevention, support and 
mitigation measures 

* More care for those with HIV and AIDS and their families 


Staff further felt that the community-based programmes meant there were less pressures 
on clinic-based services, as individuals would first seek out (free) community health 
worker assistance before going to the health centre, 
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Staff perceptions were mirrored by community members who participated in the focus 
group discussions run by the research team. Key features highlighted & communi 
members also included better access to curative services and the aaa paces 
that community-based services were making in people’s lives. Community members 
also confirmed that user fees were a major deterrent to accessing ee 
individuals who could not afford them. Community members also found that rae: ee 
attitude of many Classified Daily Employees (CDEs) in health centres was a tbe oe. ; 
deterrent to using clinic-based services. CDEs are semi-skilled staff who are often the 
first person that health clinic users encounter. At present there do not appear to be an 
programmes in place to train and support CDE level staff. j 


Community level control over health services is dependent on how motivated individual 
health units are in organising community health management committees. While 
government policy states that all centres should have management committees, 
adherence to this policy is limited. 


Organisations that work as partnerships/support advocacy: The three partner 
organisations that work primarily as networks and advocacy groups reported that 

they had achieved positive results in the areas that they work in. The Church Health 
Association of Zambia (CHAZ) has been very successful in lobbying the government on 
behalf of faith-based organisations in order to increase the proportion of government 
health funds going through mission health services. 


The Catholic Committee for Justice, Development and Peace (CCJDP) reported that its 
work has had a ‘profound’ impact on beneficiaries through its community sensitisation 
and mobilisation work. Through working with community groups CCJDP has helped 
them to be actively involved in advocacy and lobbying activities. As a result, 
communities have demanded justice on issues that affect them. CCJDP training at 
diocese level has also improved diocesan staff capacity to discern issues, generate and 
engage in debate without necessarily involving the central level. 

Overall, it was observed that there has been an increase in levels of economic literacy 
and that people are able to see how economic and other policies affect their day-to-day 
lives. Despite the positive impacts on beneficiaries registered, it was revealed in one 
interview that within the church issues of gender equity have yet to be looked at; the 
church itself is not very gender aware, while staff face a great number of stereotypes 
with regard to culture and gender relations. Often women are not in the forefront of 
activities, and their participation is hampered by social norms where target project 
communities are ordinary members of society. 

In the case of CSPR, it was reported that the organisation responds to this at two 
different levels. Issues of gender and HIV and AIDS have recently been mainstreamed 
in their Poverty Monitoring Tool. The different impact of government eee and 
programmes on the sexes as it relates to poverty is one of the organisation's core issues 
in monitoring PRSP implementation. 


7.3.4 Quality 

Organisations that provide services: It was genera 
supported health services were of a higher quality 
reasons for this include the fact that the diocese an 
qualified staff working for them than did the governm 
to have a more ready supply of drugs available for trea 
attributed to the fact that CFA partners tended to emphasise con 


lly felt by all interviewed that CFA- 
than government services. The 

d Plan services tended to have more 
ent-supported services and tended 
tment. Better quality was also 
tinuing professional 


EVALUATION OF HEALTH THeme SEPTEMBER-DECEMBER 2004 
capacity building. For example, Plan trains health centre 


including integrated management of child illness. This 
classification, treatment and more 


development and community 


staff in a number of subjects, 
was reported to have resulted in improved diagnosis, 


appropriate referrals to higher-level facilities. 


Health centre and hospital staff felt that while providers strive to provide affordable and 
appropriate quality health care, these efforts are compromised by a shortage of drugs 
and by poor staffing levels. Staffing problems are especially acute in rural hospitals, 
where it is difficult to retain medical doctors. As a result, the few that remain in active 


service are overwhelmed by the workload. 


Organisations that work as partnerships/support advocacy: The quality of the work 

of the various partnerships/networking agencies can be demonstrated through the 
effectiveness of their advocacy and training activities. As seen above, these do appear to 
be having an impact at both local and national levels. However, whilst acknowledging 
that much work is being done, respondents from these organisations also indicated that 
they needed more support to build capacity in technical analysis in order to sharpen 
skills in dealing with their programmes, as this was said to be lacking in some instances. 
Furthermore, it was observed that the rate at, and extent to, which social issues arise 
that require their involvement, coupled with limited capacity in terms of staff, often 
overwhelmed them. This indicates more effort is needed for continuing professional 
development, for setting priorities and ideally for recruiting and retaining more staff 
both within the partnerships and within their member organisations. 


7.3.5 Coherence 

Organisations that provide services: Diocesan and Plan health-related services are 

well integrated into Zambian government systems. The Church Health Association of 
Zambia (CHAZ) receives government funding to support faith-based health centres and 
hospitals, and national health planning includes the contribution made by faith-based 
organisations. Plan’s support to district health services and early childhood development 
also conforms well with the government's decentralisation strategy, where the emphasis 
is on district-level planning and service provision. Therefore, at a macro level, there is 
strong coherence between the Plan’s operations, churches and government. 


In districts where Plan operates, there is a good collaborative relationship between the 
Plan programme and district authorities. In Mazabuka, for example, it was pointed out 
that whereas Plan has the capacity and resources to build health centres, the district 
health office had to be consulted. The DHMT ensures a fair geographical distribution 
of health centres and is also responsible for providing staff, drugs, technical and 
supervisory support to the health centres once they are built. 


District-level government staff felt that relations were not strong with the diocesan 
services reviewed by this evaluation. Dioceses are invited, along with other key 
stakeholders, to participate in annual planning cycles at district level, as well as to 

take part in reviews of district health action plans. In tum, DHMTs are invited by the 
dioceses to take part in their own planning and review activities. However, DHMTs felt 
that they were often not informed about all of the health-related funding of the dioceses 
and that there were still problems with coordination. DHMTs stated that the District 
Health Office only knew the amounts that faith-based facilities receive as government 
grants and often did not know the overall financial position of these partners. As a 
result it is difficult for district offices to clearly know what these partners are and are 
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not able to do, especially in terms of targeting, 
Furthermore, district staff criticised diocesan he 
support family planning services, which are ex 
in their catchment areas. As a result, district h 
despite their own weak logistic and resource c 


monitoring and evaluating their services. 
alth services for not doing more to 
Plicitly requested by community members 


ealth officers have to make up for this 
apacities. 


Diocesan health managers admitted to the research team that they are not always 

open and transparent about their activities with their respective DHMTs as mire 

staff already perceive church health services to be more wealthy than government 
services, and if the scale of funding for diocesan programmes were revealed this 

would only reinforce the perception and lead to a reduction in government support for 
their services. There was also a feeling that relationships with the district are highly 
dependent on the quality of individuals within the DHMTs, and the willingness of 
DHMTs to interpret government policy in a more flexible manner. For example, some of 
the diocesan health services had managed to secure housing allowances for their staff 
from DHMTs, as these particular DHMTs were willing to take a broader interpretation of 
housing allowance policy. 


The above discussion needs further reflection on how faith-based organisations and 
other non-state providers interact more fully with decentralised authorities in countries 
that have gone down this route. Longer-term sustainability and scaling up of services 
are highly dependent on how well faith-based programmes can cooperate and integrate 
with government services. 


Coherence with CFA policy is also somewhat mixed. Where gender is concerned, some 
of those interviewed in the Cordaid-supported projects admitted that the Catholic Church 
has not taken proactive measures to address gender inequity and that the issue did 

not even arise in planning discussions. As one of the staff said “As a diocese I do not 
remember us saying let’s look at gender inequality or programmes that will address this”. 
The research team did find that there were some efforts made to ensure that women 
were included in community structures, but not within diocesan management structures. 


As far as services adopting a stronger ‘rights-based approach’ is concerned, there is little 
evidence that this approach has been fully integrated into the thinking and processes 

of partner programmes. In the Zambian context, ‘demand-led’ services would probably 
make services free at the point of delivery, with much higher community involvement 
in health service management than is currently the case. Neither of these are being 
actively promoted by CFA partners. On the other hand, community-based services would 
appear to be very much community owned and receive a great deal of support through 
community volunteer activity. Even in these cases, though, it is unclear the degree 

to which community members are actively involved in claiming their rights to such 
services, nor the degree:to which CFA partners encourage them to do so. 


Organisations that work as partnerships/support advocacy: CHAZ’s work is fully 
coherent with both government and CFA policies, though implementation through 
its members is more variable (as seen in the discussion above). CCJDP and CPRS are 
working more at the cutting edge of government policy, helping to influence policies 
that promote greater equity and social justice. It was also pointed out that because 
of the organisation’s voice for the poor, the government was challenged to meet its 
responsibility to the poor. Partly as a result of CCJDP’s advocacy, the government 
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started being more vocal about the poor and improved resource allocation for poverty- 


reduction activities. rem 
CSPR has made an impact in its work through generation and dissemination of 


information aimed at poverty reduction. It was reported that CSPR is often consulted 
in the development of various international agencies’ Country Assistance Strategies, 
despite the difficulties in accessing information from various government stakeholders. 
Other constraints include the lack of a concrete framework within which civil society 
can operate easily with government, since the government was not obliged to involve 
civil society organisations in decision making. In other cases, it was reported that 
organisations’ advocacy was perceived as a threat to the government, raising anxieties 
for certain civil society organisations that do not wish to engage in an open conflict 


with government institutions. 


7.3.6 Participation | . . 
Organisations that provide services: Beneficiary involvement in health services planning 


and management was reported to be more pronounced at health centre level than 
hospital level, with most participation occurring at planning and implementation stages. 
It was observed by government staff that popular structures such as the Neighbourhood 
Health Committees (NHCs) and Health Centre Committees (HCCs) consult ordinary 
community members on community health concerns and priorities through community 
level meetings. However, the extent to which committees are active varies from place 
to place and depends on the calibre of staff in the health facilities themselves. In 

some places it was reported that the level of community participation is reduced to 
only providing labour or raw materials when building or rehabilitating the health 
infrastructure. Wider community representation on hospital boards is absent and was 
attributed to the referral role that hospitals serve. 


Programme level staff reported that while attempts were made to involve communities 
in planning and managing services, there was often a subtle reluctance on the part of 
community members to become actively engaged in such activities; they say that if 
they are happy with what the programmes are doing, why should they be involved. 
Despite this, some examples of good practice are worth highlighting. In Ndola Diocese, 
staff, doctors and community members and representatives drawn from Hospital Health 
Boards and NHCs meet annually to share experiences and discuss developments. This is 
in addition to progress reviews that are held four times a year by the Diocesan Board. 


In Plan areas, community-based agents working under the supervision of 
Neighbourhood Health Committees are active in community mobilisation. In Mazabuka, 
community members are involved in problem identification, decision making at different 
levels, including involvement in health centre related structures and in developing 
community development plans. Community members in Plan programme areas also 
reported that, as a result of health interventions in their communities, community 
members have felt more empowered to deal with issues that concern them through being 
linked to MOH structures. Plan staff provide capacity building for community members 
so that they can engage more effectively in community level committees and this would 
appear to have had a significant impact on community member confidence. 


Focus group interviews with young people found that Zambian youth are generally 
excluded from involvement in community management activities. Younger people have 
been barred from joining committees by their elders, who tell them that as they are 

not yet married they have nothing to contribute. This is a great source of frustration 
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for young people in the programme areas, tho 
problem is a general one across the country, 
are dominated by elders. Even in Plan Zamb 


more active engagement with youth to encourage their involvement in managing 


project activities, young people still have no representation on Neighbourhood Health 
Committees or Community Development Committees. 


ugh organisation staff admit that this 
where all types of community activities 
1a programme areas, where there is much 


Organisations that work as partnerships/support advocacy: Participation of members 
in the various network organisations is high. As far as the church-based organisations 
are concerned, the various dioceses and missions are actively involved with CHAZ in 
supporting lobbying activities, while the diocesan community-based activities have 
provided an ideal vehicle for CCJDP’s training programmes. CCJDP works primarily 
through parish committees at community level, providing training the trainer activities 
in subjects such as Justice and Development or Mobilisation for Social Action. 


CSPR involves beneficiaries through Provincial Civil Society Groups, which represent 
community organisations. By working with members who have contacts at local 

level, CSPR has helped increase community ability to ask or raise questions. This has 

a deterrent effect on misuse or misdirection of resources by leaders or indeed any 
service providers. For example, pressure on Members of Parliament (MPs) to have more 
interaction with the electorate resulted in opening of constituency offices in a number 
of constituencies. 


7.3.7. Sustainability 

Organisations that provide services: Concerns about financial sustainability raised in 
both the DRC and Nepal evaluations hold true for the Zambia partners as well. In some 
ways financial sustainability of church-based services are better assured now that they 
receive some proportion of government health funding, though this is still far too low to 
cover the actual costs of running the health services. Efforts by health services to raise 
more income through user fee charges have proven to be not very effective. Moreover, 
where user charges are strictly administered, they are proving to be a barrier to health 
service access for the poor. 


Other aspects of sustainability are more positive. Government stakeholders felt that the 
substantial capacity-building activities undertaken by CFA partners helped to increase 
service quality and assure sustained improvements in service delivery. The only negative 
side to the capacity building and training work cited is the fact that many of the staff 
who benefit from more comprehensive training programmes often use this to then seek 
better paid jobs elsewhere, especially outside Zambia. 


Plan’s programme of training community volunteers has also contributed to programme 
sustainability as these volunteers then report to and are supervised by es aren 

and health centres. Community mobilisation activities are complemented by training 
community groups in community partnerships and project sustainability. In Mazabuka, 
for example, community management committees have taken over the management and 
maintenance of the water and sanitation infrastructure put in place by the Plan project. 


Organisations that work as partnerships/support advocacy: Sustainability of the various 
network organisations is found more in the reach of their activities and the degree to 
which their training and advocacy work is leading to changes within ace — 
organisations. The evaluators found that CHAZ, CCJDP and sata eo 


84 EVALUATION OF HEALTH THeme SEPTEMBER-DECEMBER 2004 


er organisations. Sustainability could be further 
acity building of member organisations, especially 
hange agenda they are confronted with. 


to lasting changes within their memb 
enhanced through more efforts at cap 
those that feel overwhelmed by the social c 


7.3.8 Added value } 
In Zambia, the contributions of Cordaid and Plan Netherlands to their partner 


organisations have had a significant impact on programme performance. Financial and 
other forms of support extended to these organisations make it possible for planned . 
activities to be implemented in a timely and appropriate manner. For example, Cordaid 
support to different health programmes helps to complement funding from elsewhere, 
ensuring that much-needed resources are available for a wider range of services. On 
top of this, Cordaid supports training of personnel and retention schemes for medical 
doctors, which are much-needed incentives to maintain staffing levels. 


It is also evident from the findings that where there is little expertise, building of 
technical capacity amongst staff is required for effective planning and implementation 
of programme activities. Plan Netherlands’ support to Plan Zambia in this area over the 
years has resulted in the scaling up of activities in programme areas. 


There are also many challenges to the partnership between the CFAs and their Zambian 
counterparts. Organisations receiving Cordaid funding are very appreciative of the 
long-standing relationship they have enjoyed with Cordaid and its predecessors. They 
are also aware of the pressures Cordaid staff in the Netherlands are under to respond 

to government demands for results. However, Cordaid’s partners noted that the 
transformation undertaken by Cordaid felt somewhat rushed from their perspective, with 
a resulting loss of sight of how things are actually working on the ground in Zambia. 
There is a sense that Cordaid staff are not aware of the capabilities and constraints 
under which their partners operate, and that they make unrealistic demands for results 
in too short a time frame. 


One particular area of concern highlighted by partners was the fairly bumpy 
communications between Cordaid in the Netherlands and their Zambian partners. High 
staff turnover at Cordaid was identified as having contributed to a loss of understanding 
of partner perspective, while also forcing partners to resubmit information that they had 
already sent to a previous post holder in the Netherlands. 


Service delivery partners were also critical of the short rotation of Cordaid interns 
working with Zambian partners. Having interns coming for just a few months does 

not create continuity, as individuals are being replaced just at the time when they are 
beginning to get used to the system. Interns also arrive with very weak knowledge of 
the partner organisation they have come to work with and have unrealistic expectations 
of how much can be accomplished. 


For networking partners, according to the findings, Cordaid’s support to CHAZ, CCDJP 
and CSPR has been more in terms of financial support. This has contributed to scaling 
up of activities to sub-national levels. More intervention from Cordaid especially in 
supporting skills development in these organisations is essential for improving the 
technical capacities in these organisations. 


Finally the restrictive funding process imposed by Cordaid was raised as a pressing 
concern for partners. The partners noted that they feel they are better able to judge 
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which programme areas are more urgent than others and therefore need more attention. 
As such, it would be better if Cordaid could just send a lump sum of funds without 
restricting how much should go to which area. When partners do use funds to cover 
urgent needs but outside the stipulations of Cordaid they are met with punitive actions. 
In response to this, Cordaid staff insist that their contracts are highly flexible. In this 
case, there is an apparent communication problem that needs resolving so that Cordaid 
and its partners are more in accordance about their contractual relationship. 
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8.1 


Analysis and Discussion 


In this section, the report responds to the evaluation 
based on the results presented in Sections 4 through 
all had very different contexts, from Zambia’s very stable political context to DRC’s 
highly unstable and conflict-torn situation. In Nepal, conflict is also affecting parts 

of the country. All three countries have poor socio-economic indicators and health 
systems that struggle to provide adequate services for their populations. As such, the 
role of non-state providers in all three situations is of vital importance to the health of 
communities. In countries in conflict, such as DRC and Nepal, non-state providers such 
as those supported by the CFAs lend a sense of normalcy to very abnormal lives, by 


bringing communities together to work for their own improvement, while also providing 
much-needed services. 


questions developed by the CFAs, 
7. The three countries evaluated 


All three organisations covered in this evaluation put much emphasis on ensuring 
community participation, either by facilitating community management of project 
activities (e.g. ECCDs for Plan) or through engaging community members in various 
actions (e.g. health promotion work by community groups in DRC or Zambia). The 
partners supported through the CFA’s work are also important parts of the development 
processes in their respective countries as they are made up primarily of highly motivated 
national staff. The evaluation found many positive elements to the health-related work 
being done by partners, as well as several challenges. Both successes and challenges are 
highlighted below. 


Which visions, strategies and activities do partner organisations have to 
improve access to, use of and control over basic health (related) services by 
potential users? 


All organisations evaluated in all three countries have clear mission statements and 
objectives relating to improving the lives of poor people, either directly through health 
service delivery, or through health promotion and networking activities. While it was 
impossible to measure the quantitative impact of the various partners’ work, it was clear 
through interviews with various stakeholders, staff and community members that the 
various partners were improving services, and as such, achieving their organisational 
objectives. This was true for both service delivery and partnership/networking 
organisations. 


Few of the service delivery organisations have explicit objectives to improve or monitor 

beneficiary access to or control over basic health-related services; they see this as being 

mediated more through existing national policies that promote some form of community 
health management committees. Furthermore, there is currently no means of oe 
the involvement of poorer sections of the community. Anecdotal evidence indicates that 
it is the better off in most communities who participate in community committees. 


One area of concern raised in the evaluations in DRC and Zambia was the i went pg 
or objectives for assuring gender equity or increasing the role of women in Catholic 
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gnising that there is a prevailing male-dominated 
diocesan funders such as Cordaid could and should 


do more to challenge norms that disadvantage women. Only Plan projects have specific 


ctives for improving children and young people’s say over services targeting them, 
ommunities again limit the degree to which 


diocesan programmes. Whilst reco 
culture within the Catholic Church, 


obje 
though cultural and social norms in Plan c 
they are involved in community decision-making processes. 


It is unclear how much beneficiaries are included in the strategic planning activities of 
partner organisations, except through their local government or parish representatives 
for most of the service delivery partners. In general Cordaid and ICCO partners 
interviewed did not see the relevance of community participation in management of 
health services, especially above primary care level. Beneficiary participation was seen 
as more of a conduit for improving user fee collection or extending health promotion 
activities. The exception to this is Plan in both Nepal and Zambia, which actively 
engages community members in developing local community development plans, which 
Plan then helps them to implement, and which feed into Plan’s country strategy cycle. 


Questions were raised by a number of beneficiaries interviewed as to who participates 
in the various committees and meetings, with concerns raised about poor levels of 
participation of both women and youth. As all the CFAs are increasingly advocating 

for a shift to a rights-based approach in the work they fund, the question of how 
beneficiaries can influence even the partner organisations themselves needs to be 
addressed more explicitly, while also addressing the problems of engaging more socially 
marginalised groups within communities. 


Community participation in partner organisation activities is reasonably strong, though 
this evaluation found much could be improved. There is a large difference, for example, 
in how the different BDOMs supported by Cordaid in DRC approach communities 

and support community mobilisation activities (e.g. BDOM Bukavu versus BDOM 
Kananga), which in turn affects the degree to which community members engage 

in health management committees and other actions for improving their health and 
health services. In Nepal, active community engagement was found to be an important 
element in keeping programmes running, as the insurgents in particular will not bother 
organisations that they see as responding to the expressed needs of communities. In 
Zambia, as with DRC, community involvement in health management committees was 
highly variable between the dioceses. However, there is high community engagement 

in community-based health activities, such as home-based care and Plan’s ECCDs. 

The research teams in both Nepal and Zambia were impressed with Plan’s community 
mobilisation efforts, which are assisted by Plan community development facilitators. The 
degree of volunteerism in those communities is markedly high, and community members 
who were interviewed expressed great enthusiasm for the work they were doing. 


Networking/partnership organisations are member driven, and therefore have high 

levels of engagement of members in their strategic planning processes. The degree to 
which beneficiaries feed into this process is highly dependent on the members of the 
partnerships. Some of those working in the secretariats of partnership organisations 
cited numerous challenges to maintaining high levels of participation and engagement 
from members, as each member organisation brings to the table its own set of visions, 
objectives and ways of working. This can create problems for implementing strategies in 
support of improving health-related services. In some cases CFA partners who have been 
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labelled as a partnership or networking organisation are in fact rather large local NGOs 
that implement their programmes through community organisations (e.g. WOREC). 


The visions, strategies and activities of al] partner organisations evaluated either 
represented an excellent fit with current government health policy (e.g. many of the 
partner organisations help to implement government health policy in ‘ihe collaboration 
with Ministry and district level officials) or were on the cutting edge of policy making 
so that their own activities were helping to break new ground in government policy 7 
(e.g. the work of Centre Olame in DRC, YUHP and Plan in Nepal and Ndola Dioceses’ 
HIV/AIDS work in Zambia). All organisations subscribe to features of community 


participation, which have also been mainstreamed in national Poverty Reduction 
Strategies in all three countries. 


The diocesan programmes in both DRC and Zambia work within the decentralised health 
system of both countries, and in the case of DRC, even oversee the management of 
certain decentralised areas. Similarly, BNMT in Nepal is providing health services, often 
on behalf of the government, in the eastern region of the country, while smaller groups 
such as CCBN make smaller-scale contributions. All stakeholders saw the faith-based 
organisations as vital parts of the national health system. The training programmes 
supported through organisations such as ESP and IPASC in DRC, and through the 
dioceses in Zambia, were also seen to fit well with national priorities and programmes. 


While in general the CFA service delivery partners are working in line with government 
policies on basic health care provision, one area of concern raised was the Catholic 
Church’s stance on family planning and HIV prevention. In communities that are 100% 
reliant on Catholic diocese facilities access to family planning and condoms is extremely 
limited. 


The level of coherence between the partner organisations’ vision, strategies and 
activities and the policies of CFA is more variable. The weakness of efforts to assure 
gender equity in a number of the Catholic dioceses in both DRC and Zambia was flagged 
by both research teams. Understanding and ownership of rights-based approaches 
amongst service delivery partners is also weak, except in Plan programme management. 
Numerous partners asked the evaluators for clarification about what is meant by a 
‘rights-based approach’ and how they were supposed to change their work to shift in 
this direction. Understanding of RBA remains poor even among Plan’s field-based staff. 
It is evident that, while partner organisations have received the message from CFAs that 
CFA policies and strategies are changing, they haven't been adequately supported to 
understand how they can respond to this shift, even when elements of their programmes 
already conform to new CFA strategies. The following quote from Zambia illustrates the 
frustration partners feel in meeting what are perceived to be unrealistic demands from 
their CFA partner: 


We are involved in so many activities. We are not where they think we are in terms f 
capacity and technical know-how. They might talk about a strategic plan. First of a : =e 
do not know it. Before we start sharing ideas, they ask us how far we have gone with the 


process. 


The research teams in Nepal and the DRC also reflected that the whole subject of 
‘claiming rights’ in the current context of conflict presented understandable challenges 


to partners. In both countries government bodies are barely in a position to respond to 
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nities, and in Nepal local government staff have to be careful in 
ations so as not to compromise 


dered thought to how to 


the needs of commu 7 . 
how much they interact with community-based organis 


their activities. It is important that the CFAs give more cons! 
apply RBA in conflict and post-conflict situations. 


For the partnership organisations evaluated, their vision, strategies and activities are 


much more related to advocating for pro-poor changes to government policies and 
therefore have a stronger fit with CFA concepts of rights-based approaches. Many are 
dedicated to helping their members, and the communities their members represent, to 


claim their rights. 


Those partners whose vision, strategies and activities feature a strong element of ' 
community ownership and engagement are likely to find they have the most sustainable 
impact on their beneficiaries. Where the emphasis is on capacity building (through 
literacy, empowerment to lobby or self-help to respond to crises) partner organisations’ 
activities are having a lasting effect that should survive even if the organisations 
themselves cannot. Other organisations, such as UMN’s YUHP programme in Nepal, 
have built in the handover of activities to local government from the beginning of the 
project, ensuring that health services will continue to be delivered beyond the life of the 


project itself. 


Within the CFAs themselves, programme coherence varies tremendously . Plan 
programmes have the clearest connections between international organisational 

policy and its implementation on the ground. This is expected as it is an operational 
organisation, with staff down to field level working on common aims and strategies. 
Cordaid’s programmes also show reasonable coherence between international policy 
and implementation, due primarily to the fact that it works mostly through Catholic 
structures within countries, and supports umbrella organisations, such as CHAZ, which 
serve to link partners’ work together and advocate on their behalf. 


ICCO has the least demonstrable programme coherence, as its approach has been to 
select partners that fit with ICCO policy priorities but that may have little connection 
to each other. As such, the synergistic impact ICCO has through its partners remains 
weak and fragmented, even though the work of individual partners is laudable. All 
three approaches are valid, as it can be argued that ICCO’s approach gives maximum 
ownership of development processes to its partners. The critical question therefore 

is what level of health impact does each CFA wish to achieve, and does its current 
approach allow it to achieve this? 


8.2 What are the qualitative and quantitative results of these programmes and 
for whom do they improve sustainable access to, use of and control over 
basic health (related) services? 


In all three countries CFA partners were on track with meeting annual activity targets, 
where they existed. In most cases, though, it was virtually impossible to quantify 

the impact of the various programmes, due to a lack of indicators and data within 
organisations and programmes. Partner organisations either expressed the need 
themselves to be supported in developing objectives and indicators, or the evaluation 
team members indicated that this support is much needed. 
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is that they exist to respond to 
did not need impact indicators to measure their effectiveness 


Plan’s Early Childhood Care and Development Centres are often the only community- 
based health support available to women and young children in those areas mee 
organisations such as BOAD, CME, the Centre Olame in DRC, CVICT in Nepal the YUHP 
programme in Kathmandu and the HIV and AIDS home-based care programmes in 
Zambia are providing essential services to very vulnerable people that no one else is 
providing at present. These have a significant impact on vulnerable people's well-being 
in the programme areas. 

At present it is impossible to know to what degree most of the CFA partners are in fact 
reaching the poorest and most vulnerable in the communities where they intervene, as 
there are no monitoring systems in place that specifically examine this. 


There is no apparent link between how long organisations have been supported by the 
CFAs, the amount of resources committed by the CFAs and the overall effectiveness or 
impact they are having. There are a number of factors that make such an analysis highly 
suspect, including the fact that most of the longest running partnerships for ICCO and 
Cordaid are well established institutions in their countries (e.g. mission hospitals and 
international NGOs), with numerous sources of funding other than the CFA. Institutional 
capacity has existed in these organisations for a long time, and is likely to have been 
the main determinant in their becoming a CFA partner in the first place. Given how 
long some of these relationships have existed (20 years or more), it seems that ICCO and 
Cordaid have not been sufficiently challenging or critical of their partners, especially as 
the CFA’s own policies have evolved. This is especially true with respect to the Catholic 
Church’s attitudes and practice related to gender, sexual and reproductive health-related 
issues. 


This evaluation has found that there are a number of deterrents to poor people's 
access to and use of services provided by partner organisations. The key ones cited 

by beneficiaries were user fees, poor levels of staffing in health units and the hostile 
attitudes of some health care staff. Research into health service access in developing 
countries concurs that user fees and poor service quality are major obstacles to poor 
people’s access to services’. Research has also shown that developing and transitional 
economy countries that do not charge user fees have substantially more — 
health systems, and have created a far greater sense of social responsibility. Pressure 
from citizen’s groups and donors has led to the abolition of user fees for health care in 


various countries. 


7 Coalition for Health and Education Rights (2002) User fees: the right to education and health denied. 
London: CHER 

* Rannan-Eliya R and Somanathan A (2005) Access of 
the role of health systems from Equitap. London: DFID 


the Very Poor to Health Services in Asia: Evidence on 
Health Systems Resource Centre 
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erned about improving access to and use of health services 
they need to explore with partners and their government 
ess for the poor and the very poorest. There are a 


If CFAs are genuinely conc 
for poor and vulnerable people, 
stakeholders how they can improve acc 
number of strategies available for doing this, including: 


e Health equity funds: Facility-based funds, managed by an organisation independent 
of the facility, that provide financial support to cover user fees, transportation costs, 
food and basic items while undergoing treatment and social care costs? ; 
Means-tested waivers: Families identified as living below the national poverty line 
are provided with health cards that entitle them to free care” ey 
Cash transfers: social assistance schemes where funds are paid to poorest families tied 
to certain conditions, including children’s attendance at school, women’s attending 


antenatal consultations, etc." 


The CFAs need to also examine the strengths and weaknesses of current efforts to make 
services more accessible to the poor, such as government-run social insurance schemes, 
facility-based mutuelles and cross-subsidising from other programmes, as is done by 
BDOM Bukavu with its generic drug production. 


As far as control over health-related services is concerned, the greatest community 
engagement was found in diocesan programmes of home-based care and in Plan’s 
ECCD programmes, where there is heavy reliance on volunteer support and community 
committees. In both cases, community members are provided with training and 
support from programme staff to help them have the confidence to engage more fully. 
Community health management committees and other community-based structures 
that have been set up to help with managing health units are extremely variable in 
their effectiveness, and even in their existence. Whether these have been set up and 
the degree to which they are effective is very dependent on the staff of the health units 
the committees are supposed to manage. This mirrors experience with community 
involvement in health management committees in other countries. 


What is less clear is the degree to which partnerships are encouraging the scaling up 

of successful activities of their members, or are acting as vehicles for the spread of 
innovative practices. In the DRC, different BDOMs appear to have highly divergent 
strategies for addressing similar problems and are not learning from each other. It is also 
unclear how well CONAFED is linking the activities of its various members, or is using 
the potential strength of its partnership to advocate more strongly for women’s rights. 


In general the quality of care provided by different partners can be considered to be 
technically good or very good. Quality is determined by factors such as the quality 
(and presence) of qualified staff, staff attitudes to patients, and levels of material, drugs 
and equipment available to diagnose and treat patients appropriately. All services 
evaluated rated well in terms of material, drugs and equipment in health units. Partner 


* Hardeman W, Van Damme W, Van Pelt M, Por I, Kimvan H and Meessen B (2004) Access to health care for 
all? User fees plus a Health Equity Fund in Sotnikum Cambodia. Health Policy and Planning 19: 22-32 

° Bitran and Giedion (2003) Waivers and Exemptions for Health Services in Developing Countries. Social 
Protection Discussion Paper Series No. 0308. Washington DC: World Bank 

' Nigenda G. and Gonzalez-Robledo L (2005) Lessons offered by Latin American cash transfer programmes: 


Mexico's Oportunidades and Nicaragua’s SPN. Implications for African countries. London: DFID Health 
Systems Resource Centre 
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organisations also indicated their a 
development activities for clinica] 


There remain, however, serious challenges to service quality in all three countries, The 
conflict and weak state structures in DRC and Nepal mean there are severe problems 
with recruiting and retaining qualified staff in partner health units. Zambia also faces 
serious problems with retaining staff in remote health units, whether in the state or non- 
state sector, and these problems are further compounded by the loss of many trained 
Staff to AIDS. Front-line, semi-skilled staff in health units, who are often the first person 
encountered by patients, are not benefiting from appropriate training and development 
opportunities, and yet can determine whether a patient continues to use that service or 
not. 


In terms of impact on country policy and practice, Plan’s ECCD programmes in both 
Nepal and Zambia, and its health programme in Zambia, have been influential with 
government policy on child health and early years development. In both countries, 
national and local government staff expressed high appreciation for Plan’s efforts to 
work with, support and train government staff alongside Plan’s own programme staff. 


Many of the organisations evaluated are also working on the margins of existing policy, 
and could therefore provide models for how similar work could be approached in future. 
Key examples of this included Centre Olame, BOAD and ELIMU in DRC; CVICT, CBRS 
and YUHP in Nepal; and Diocese of Ndola’s HIV and AIDS programme and CPRS’s 
support to civil society structures in Zambia. In some cases, these organisations are 
already well networked and are influencing the development of national policies. In 
other cases, such as CBRS or Centre Olame, the activities remain at a very local scale 
with little prospect for their model being adopted elsewhere, as much as the services 
they provide are needed. 


The evaluation also found some important areas for improving the links between partner 
organisations and local/national government structures. In Zambia, with all government 
health budgets targeting the faith-based health services passing through CHAZ at 
national level, there is little incentive for dioceses and missions to interact with district 
health teams. In fact, diocesan staff cited a number of disincentives for working in a 
transparent manner with their district counterparts. While acknowledging the very real 
challenges facing districts and dioceses, the long-term sustainability of health services 
from both levels is dependent on their working closer together. Some tensions were also 
found between BDOMs (particularly Kananga) and provincial health inspectorates in 
DRC. 


While these tensions exist, the models for state/non-state cooperation in the health 
sector offered by both Zambia and the DRC provide the best possibility for longer-term 
sustainability and coherence within the health sector. In both countries the national 
programme contracts the larger non-state providers to provide services, and in the case 
of the DRC, manage health services in a geographical zone, with the government (or 
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ate provider budget. These sorts of arrangements 
as more attention is given to rationalising the 
th sector. Cordaid and ICCO need to discuss with 
ffectively with these contractual arrangements. 


its donors) contributing to the non-st 
are developing in several countries now 
contributions of all providers to the heal 
their providers how they can work most e 


For network and partnership organisations, CPRS provides an interesting example of 
being coherent with national poverty reduction policy, while also being at the cutting 
edge of ensuring that this policy is well implemented throughout Zambia. CONAFED 

in DRC works at the margins of national policy, as the DRC does not have policies in 
place that protect and promote women’s rights. The DRC research team felt strongly that 
CONAEFED could be doing more to advocate for women’s rights with government and 
especially within the larger civil society network. 


A major area of concern mentioned in both the DRC and Zambia studies was the need 
to increase the involvement of women in planning, implementation and management 
of health-related activities run by faith-based organisations. Attempts are made to 
involve women in health promotion and caregiver-support activities, and some health 
committees have reasonably equal representation of men and women. However, women 
remain excluded from the management structures of many of the diocesan and mission 
structures. A similar picture emerges for involvement of young people. In Zambia, 
young people are engaged in various project activities by Plan but youth groups 

also expressed frustration at not being given more of a role in various community 
committees in their areas. It would appear that a ‘glass ceiling’ exists that prevents 
women and youth from participating beyond certain levels of organisational and 
community structures. 


Some of the more innovative programmes have yet to develop a real community- 
based constituency. These include the Centre Olame in DRC as well as CVICT and CBRS 
in Nepal. Many of these initiatives are still fairly young, and staff are overwhelmed 
with demands for their services. As such, the organisations would benefit from greater 
support from their CFA partners in developing strategies for greater community 
involvement and ownership. 


Network and partnership organisations by their nature are participatory and highly 
dependent on their members for being able to implement activities. As shown above, 
CCJDP and CSPR in Zambia have been able to be effective through working with 
community-based organisations that form parts of their network, while also providing 
much-needed capacity building of community groups. 


Across all countries, financial sustainability of partner activities is extremely weak, and 
will continue to be so for the foreseeable future. This is not surprising in such aid- 
dependent countries, nor in countries such as DRC and Nepal, which have to contend 
with conflict and the threat of conflict. As seen in the discussion above, some efforts 
have been made to ensure better financial sustainability through employing user 
charges, but the disadvantages of user fees far outweigh the advantages they bring. 


Sustainability of programme activities, especially those that are community based, 
appear to be more certain in those programmes that actively engage community 
members, that encourage self-help initiatives and that build community capacity 
to carry on beyond the life of the project. Examples of programmes that have high 
prospects for longer-term sustainability through community mobilisation include 
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= S work, ELIMU's literacy programme and the various home-based/community- 
| ased Care initiatives supported by CFAs. Strong governance arrangements could also 
improve chances for sustainability, as shown by CBRS in Nepal 


Sustainability can also be better assured by integrating programme activiti 

into government systems, as is being done with Plan's programmes and YUnP 
Kathmandu. However, as can be seen with government plans to expand the mi 
programme in Nepal, public resource constraints may limit the government's capacity t 
implement programmes to the standard set by non-state providers. ee bs 


The sustainability of the network and Partnership activities seems more assured than 
some of the service delivery programmes, due primarily to the reach of these networks 
and the capacity-building work that is being undertaken to support network and 
partnership members. While membership subscriptions can help to provide a certain 


degree of sustainability, these partnerships and networks need to also ensure they have a 
diverse funding base to continue their work. 


To which extent do complementary partnerships, participation in networks 
and/or alliances contribute to better performance of partners? 


Many of the networks and partnerships evaluated have strong elements of capacity 
building of members and community organisations as key strategies for enhancing the 
work of their networks. The following examples illustrate the importance of capacity 
building for improving the performance of partners: 


¢ CONAFED has developed and runs training modules for members in raising 
awareness of gender issues and supporting the mobilisation of women. 

e ELIMU has developed literacy training modules and management training 
programmes for community-based women’s groups, which directly benefits the 
performance of these groups when applied to their income-generating activities and 
other areas of work. 

e Plan’s programmes in both Nepal and Zambia have worked closely with and 
supported the training of local government counterparts, leading to improved 
performance in such areas as Integrated Management of Child Infections. 

e BNMT’s collaborative work with government staff has built government capacity to 
plan and manage TB programmes. 

e¢ CCJDP and CSPR, through training programmes aimed at members of their own 
networks, have built the capacity and confidence of community-based organisations 
to monitor and hold to account government officials. 


Where partnerships and networks are working well, as in the cases cited above, there 

is a good chance for sustainability and high level of added value. The research teams 
also noticed many areas in which partnerships and networks should be working closely 
together, but are not. For example, the dioceses in both Zambia and DRC operate similar 
structures and within similar systems. However some have been more innovative and 
responsive to community need than others, and yet there seems to we little sharing or 
spread of good, innovative practice. Another example was found with the organisations 
that had branched off from UMN projects in Nepal, where again, there appeared to 

be little sharing of good practice (e.g. in terms of governance), which could help 
organisations to improve their performance. 
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The effectiveness of the partnerships is highly dependent on a variety of ote a 
In Zambia, CHAZ has been highly effective in lobbying for more neansgpee be 
to be channelled through CHAZ members for health service delivery, but i we 

a role for itself in mediating between dioceses and their district counterparts. 

makes very effective use of the parish networks of the Catholic Church to dlniaasae 
its programme of social action and justice. In Nepal, UMN has successfully spawne 

a number of small local organisations that provide community-level health services. 
Partnerships that function more as umbrella NGOs (WOREC, BOAD and ELIMU) are 
effective in transferring funds and providing training for local community organisations 
that they support. However, these do not operate as real partnerships or networks where 
community organisations have an equal say in partnership management or strategy. 


8.4 With what results do (complementary) partnerships, participation in 
networks and/or alliances contribute to a better performance of the partner 


organisations and the health system in general? 


All organisations assessed in this evaluation are part of some type of partnership or 
network, whether formally or informally. The results that can be attributed to working 
through partnerships, be it with government partners or within a network, are as 
follows: 


e Avoiding duplication of services and effort: Good coordination between faith- 
based organisations, Plan programmes and the government has helped to decrease 
redundancy in service delivery and improve efficiency. This has allowed different 
parties to these partnerships to share resources more effectively. 

e Increased profile and awareness of issues of interest to the partnership: CONAFED’s 
work on gender equity has helped raise the profile of women’s rights in DRC, even 
though there is still some way to go. Their work has also raised awareness of HIV and 
AIDS within their membership. CHAZ has become a powerful voice in discussions 
on Zambian health policy as it represents a group of organisations that together 
make up over one-third of Zambia’s health care delivery system. CPRS’s network has 
increased awareness amongst its members of government poverty reduction policies 
and has set in place the means of monitoring how well government is doing as a 
result of its activities. 

e Increasing the potential sustainability of programme activities: some of the 
programmes evaluated are implemented through partnerships in order to ensure 
smooth handover and community ownership. For example, the YUHP programme in 
Nepal was set up to work closely with the municipality of Kathmandu as part of its 
handover strategy. BOAD Goma works through its CBO partners to encourage greater 
community capacity and resilience in the face of a humanitarian crisis. Plan works 
closely with local government to ensure that its health-related projects are integrated 
into local government services once the project is finished. 


While partnerships and networks can achieve powerful results, they do come at a cost. 
A number of respondents indicated that meetings and working out relationships were 
time consuming and fraught with tension. However, as seen in Section 6.3., programmes 
working in isolation of potential partners fail to spread innovative practice and 
constrain opportunities for scaling up their activities. 
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team found the following examples, among 
€ made: Centre Olame appears to have no link 


oth being Cordaid partners. In Ne 
; pal, local NGOs 
supported by ICCO rarely if ever have contact with each other, even though they each 


have technical and organisational experiences that others would value. The CFAs state 
that they do not wish to impose new networks on their partners. On the other hand. if 
the organisations are not broughttogether they are not given the opportunity to aie 
whether they wish to carry on meeting with and learning from other CFA partners. 


of scaling up small-scale intervention 
organisations are not as well network 
innovative activities. The evaluation 
others, of where connections could b 
into CONAFED in the DRC, despite b 


There is a particularly serious gap in partnership development amongst DRC faith-based 
organisations that needs concerted effort to fill. At present there is no organisation 

that serves the same function as the Church Health Associations in other countries, 
which help to mediate between the state and faith-based organisations, and serve as a 
quality assurance body and conduit for funding to members. In the longer term, such an 
organisation must be established, either by revitalising the SPS or through the creation 
of a new body, to ensure the spread of good practice and to advocate on behalf of faith- 
based health services across the country. 


What is the added value, in terms of the results and partnership aspects, of 
the CFAs' support for the partner organisations and their programmes? 


The first obvious answer to this question is the financial gain that the relationship with 
the CFAs brings to their partners. All partners were appreciative of the commitments 
made by their CFA donors, many of which are long term. ICCO is seen as being 
especially flexible in how it disburses it funds, allowing maximum flexibility to 
partners, within certain boundaries of the agreed programme. An observed, a significant 
added value of ICCO’s flexible, untied support is that it enabled partners to adopt and 
maintain a more holistic approach, which doesn’t address health in isolation but as an 
integral aspect of development. Cordaid and Plan are somewhat more rigid in how they 
disburse funds, sometimes setting budget lines that do not allow much flexibility, but 
which are still valuable contributions to the organisations they support. 


Partnership with the CFAs has also provided opportunities for some organisations 

to make contact with regional and international organisations that are like-minded, 
helping to stimulate ideas for work within their own programmes. For example, SSS 
in Nepal expressed high appreciation for the opportunity to attend the last AIDS 
conference in Bangkok, and the chance to network with other AIDS IEC organisations. 


The evaluation found many examples of innovative work of CFA partners, supported 
by the CFAs and other donors. Some of the more innovative work that is being scaled 
up, or has the potential for scaling up, includes Plan’s ECCD work in Nepal, the AIDS 
outreach programme in Ndola Zambia, the performance-based operations of BDOM 
Bukavu and the municipal health programme in Patan, Nepal. The added value the 
CFAs bring to these programmes is primarily the stability of their support and their | 
willingness to take on programmes that, at least at their outset, seem unconventional. 


98 EVALUATION OF HEALTH THeme SEPTEMBER-DECEMBER 2004 


eee , iati the CFAs’ support for training of 
anisations were highly appreciative of 
pa egeareaiie elopment activities. The support to diocesan 


; dies 
staff and their funding of various staff : : 
technical training in Zambia was particularly appreciated by those interviewed. Cordaid 
has also brought DRC and Zambia partners together on occasion to explain changes to 
Cordaid policy and to share experiences. However, most of those interviewed felt that 


they had received little or no organisational development support, so that they were not 
able to undertake the strategic planning and management activities that they seemed 

to be expected to do. In this case, there is some disconnection between CFA policy 
documents, where emphasis is given to the shift towards working with organisations 
and organisational development rather than with projects, and what is actually 
happening in the relationship between CFAs and their partners. 


Many partners reflected on the fact that, at least as far as Cordaid and ICCO are 
concerned, they were aware that Netherlands-based programme officers were 
overstretched with the number of countries and organisations they cover. Also, 

frequent changes in staffing within these key link posts made communication very 
difficult, especially regarding any dialogue on organisational strategic changes. To 
many partners, the organisational changes in direction outlined in the various CFA 
policy documents were perceived as more of a ‘whim’ of individual programme officers, 
as policies seemed to have changed with the individuals contacting them. The CFAs 
acknowledge that staff turnover is a problem, but one they have little control over. 

The evaluation team noted that having a Cordaid office in the DRC did help with 
communications, which is important given the particularly difficult problems of working 
the DRC. 


The results of the work of CFA partner organisations add value in a number of ways. 
Besides the obvious importance of the services provided by most of the organisations 
evaluated, some of the partners have also developed models at a small scale that have 
either been, or have the potential to be, adopted more widely. Examples can be seen in 
the work by Centre Olame, ELIMU, IPASC and BOAD, as well as BDOM Bukavu in DRC; 
by Plan’s ECCD programme and the YUHP programme in Nepal; and by the home-based 
care work of the dioceses in Zambia. 


Furthermore ICCO, and to some extent Cordaid, are missing opportunities for bringing 
real added value to their country programmes by having weak country strategies (ICCO) 
or by funding partners that have little connection with each other. This needs to be 
addressed within each organisation and with partners. 
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Conclusions and Recommendations 


Vision, strategies and activities of CFA partners 


The visions, strategies and activities of all CFA partners evaluated favour access to, use 
of and control over health services by the poor. Partner staff have a strong commiliment 
to the work they are doing for the communities they serve. However, many of the 
organisations evaluated, especially Cordaid, ICCO and their partners, rarely have 
indicators designed to help them monitor whether their visions, strategies and activities 
are in fact achieving their purpose. So, while in general organisations are achieving 


their targets (where these are specified), it is difficult to link up organisational vision 
with outputs, outcome and impact. 


Furthermore, the CFA policies for gender and rights-based approaches are not being 
translated into effective implementation strategies in any consistent fashion with CFA 
partners. Plan is the most effective at bridging this policy-implementation gap, not 
surprisingly, as it can facilitate greater understanding through its staff structure down to 
community level. Plan could do more to strengthen staff capacity to challenge societal 
norms where those norms lead to the continued marginalisation of certain groups (e.g. 
Dalit in Nepal and youth in Zambia). Cordaid and ICCO’s ways of working, while giving 
greater ownership to local/national organisations, also necessitate programme officers 
putting greater energy into communicating with partners. 


However, more thought also needs to be given to how best to encourage ‘rights-based 
approaches’ in conflict/post-conflict situations. This evaluation found that in the 
highly sensitised atmospheres of Nepal and Eastern DRC, more nuanced approaches 
are required. The DRC and Nepal governments are not able to fulfil their obligations 
to citizens claiming their ‘right to health’ In the case of Nepal local government staff 
themselves feel that their involvement in service delivery could compromise effective 
provision of much-needed health services. Rather than approaching rights-based 
approaches from a ‘duty versus claim’ dichotomy, partner organisations could be 
looking at other demand-side approaches that help build community confidence to 
claim their right to health from a wide variety of duty holders. 


Recommendations for improving health-related visions, strategies and activities 


All CFAs 

© More concerted support needs to be given to partners (government and non- : 
governmental) in organisational development capacities such as strategic planning 
to help them operationalise their goals, and assess their achievements. Expertise for 
doing this exists in all three countries studied in this evaluation, and specific funding 
should be given by CFAs to buy in this support. 

¢ In-country workshops should be held to work with partners (ICCO and Cordaid) and 

staff (Plan) on how to understand and adapt CFA core policies on gender and rights- 
based approaches (RBA). RBA in particular needs specific attention in Nepal and DRC, 
and the CFAs should look at how to improve demand-side approaches with their 
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partners, including strengthening community management of partner health facilities 


and voucher-for-service schemes that target poor and marginalised groups. 


Cordaid and ICCO res Wore ks 
e National frameworks should set out the CFA’s priorities and strategic directions 


in terms of pro-poor development and target groups, and draw out coherent links 
between its policy and the expected results of its support to existing partners, as 
negotiated with them. This latter point, of defining the CFA’s niche with the partners 
is particularly important, given that the CFA is only one among several donors for 
some partners. It would also indicate the kinds of inputs (financial, capacity building, 
support to advocacy, etc.) that partners could expect from CFAs. Such a framework 
would create a ‘common thread’ that draws an explicit line from global policies and 
strategies to implementation of those policies as agreed with partners. 

e As part of their organisational development programme, ICCO and Cordaid should 
continue to support internal organisational and partnership assessment exercises 
such as the one employed during this evaluation, to help their partners evaluate their 
own development needs. 


Cordaid 

e Given the importance of Cordaid’s global gender policy, the CFA needs to negotiate 
more robustly with its Catholic partners to ensure a much more meaningful 
involvement of women within diocesan structures. It should build on models of 
strong female leadership in the Catholic Church in other countries and explore how 
these can be adapted with its partners who work with more male-dominated models. 
As mentioned above there are examples of good practice in the case study countries 
to draw upon. However, the evaluation also found that Cordaid in particular needs 
to consider its collaboration with Catholic-run health services that are not open to 
focusing on women’s participation in decision making and that will not engage in 
effective delivery of sexual and reproductive health services in countries with high 
maternal mortality and sexually transmitted infection/HIV rates. Where church 
partners prove particularly resistant over time, Cordaid should as a last resort 
consider reducing or withdrawing support altogether. 


Plan 

e As part of developing rights-based approaches, Plan should develop a programme of 
support to its own staff, many of whom come from their programme communities, 
to build understanding of the dynamics of social exclusion and ways in which to 
challenge social norms. 


Summary of qualitative and quantitative results 


Overall, the findings of this evaluation indicate that the CFA partners assessed are 
making an important contribution to health-related services in the three countries 
studied. Whether working through partners (Cordaid and ICCO) or through direct project 
implementation (Plan), the programmes supported by the CFAs are critical components 
of the health systems within which they operate. The absence of CFA partners would 
leave a gaping hole in health-related services, especially where partner organisations 
work in remote areas of the country. Both government and beneficiary stakeholders 
repeatedly expressed their deep appreciation for the ongoing support to the health- 
related services provided by the CFAs. 
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expected results. 


Quantitative health impact is also difficult to assess 
baseline and monitoring data. New systems do not 
as national systems already exist. As the main ba 
logistical, the partners need more Support to use existing national systems while also 


considering what other data they need to collect on a routine basis in order to measure 
achievement of organisational and programmatic objectives. 


amongst CFA partners, due to poor 
need to be created to address this, 
trier to the widespread use of HMIS is 


From a qualitative perspective, health services provided by CFA partners are widely 
appreciated by government stakeholders and beneficiaries. These services are considered 
to be of higher quality than alternative (public) services. Community-based activities 
are filling a real public health need and benefit poor and non-poor alike. For many 
beneficiaries, though, user fee charges continue to be a significant barrier to their ability 
to use health services, even where costs have been reduced. Other quality issues related 
to staff retention and attitudes also represent obstacles to equitable access to services for 
poor and marginalised people. While the CFAs are helping to improve the situation with 
regards to staff retention, they are not, at present, working with partners on reducing 
the damaging effects of user fees. Those organisations that are working on more 
specialised issues, such as gender-based violence, are creating models for how services 
for vulnerable women could be developed in future at a much larger scale. 


New trends in contracting health services between the state and non-state sectors 
provide opportunities for Cordaid and ICCO’s partners to review their relationship with 
the state sector and will require shifting towards more conformity with nationally 
agreed quality assurance standards and health monitoring systems. Similar issues arise 
where Plan runs health clinics or schools on behalf of the government. 


The evaluation also found that Cordaid in particular needs to rethink its funding of 
Catholic-run health services that are not open to focusing on women’s participation 
in decision making and that will not engage in effective delivery of sexual and 
reproductive health services in countries with high maternal mortality and sexually 
transmitted infection/HIV rates. 


Recommendations on improving the quantitative and qualitative impact of health- 
related services 


’ 


All CFAs 

e All CFAs should continue to work with and strengthen their community-based 
programmes, as these are having the greatest effect on the health of poor people, and 
have greatest sustainability (e.g. Mama and Papa Bongisa, Kinshasa Diocese; Ndola 
Diocese’s AIDS home-based care; Plan’s ECCD outreach work in Zambia and Nepal). 

* All the CFAs need to reconsider their own and their partners’ strategies that depend 
on user charges for services they support. User fees are a deterrent to the use of 
services by poor people and appear to contradict the CFAs’ own pro-poor policies; yet 
they also provide much-needed resources to maintain services at local level. A first 
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uld be to hold a policy forum that includes staff of the CFAs, aati ae 
of their partners/field staff and leading researchers to help frame sie és : oe 
respond to this and how they can support or influence national policy develop 
in either eliminating user fees or creating social safety nets for those who canno 

° <c sate be provided to CFA partners (governmental and non-governmental) 
in the collection and use of routine health data to help develop an evidence base 

s to health outcomes and ensure it is well linked to the national health 


step sho 


with regard 
management information system. 


Cordaid and ICCO 
e In order to measure the health impact of CFA support, national strategic frameworks 


should include objectives with measurable indicators that reflect the aggregate results 
of their partners’ own strategic plans and targets. CFA partners also need help in 
developing capacity in this area. This will greatly support partner M&E activities, and 
facilitate future evaluations of health-related programmes. 

e Sustainability of health-related services remains a key concern for ICCO and Cordaid 
partners. ICCO and Cordaid should work with their service delivery partners on 
improving the prospects of longer-term sustainability of their activities, through 
designing handover strategies, creating more transparent links with government 
services by making better use of contracting arrangements and building on models of 
good practice for community involvement identified in this, and other, evaluations. 
ICCO and Cordaid should also continue to encourage their partners to seek ways of 
diversifying their funding base. 

e ICCO and Cordaid should examine the model of community capacity building 
employed by Plan and discuss with partners if and how this model could be adapted 
into their community-based work. 


Cordaid 

e Cordaid needs to give serious consideration to how it can support filling the gap 
left by its partners in family planning and other reproductive health services. One 
suggestion is that where the Cordaid (Catholic) partner is the sole service provider, 
Cordaid could also fund an alternative non-Catholic partner to ensure that these 
services are accessible, or that it use its influence to persuade another donor 
to provide support in this area. At the same time Cordaid should embark on a 
programme to facilitate greater cross-learning between the Catholic organisations 
it supports and negotiate more emphatically for attention to be given to sexual and 
reproductive health. 

¢ Health reforms leading to decentralisation have created a new set of dynamics that 
Cordaid’s partners in Zambia have not fully incorporated into their own thinking. 
Cordaid and its partners also need to reflect on how to build a more transparent and 
trusting relationship with government line agencies in Zambia, including being more 
open about financial flows. 


Plan 


* Plan programmes should consider exploring the UMN experience in Nepal to see 
whether an approach to the institutionalisation of their programmes could be to set 


up and support a local NGO, which would then carry on the main areas of Plan’s 
activities, 


9.3 
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Summary of e a 

— - ae ve een of, partnership impact on health-related 
The partnerships supported through CFA fundin 
to local and national policies on health for poo 
advocacy organisations also need much more t 
indicators that will help them monitor their pr 
partnerships or advocacy campaigns. Much of 
anecdotal; better data is needed to back up the 
research teams. 


§ are making a meaningful contribution 
T people. However, networking and 
argeted support to develop objectives and 
ogress in achieving the purpose of their 
the impact reported in this evaluation is 
Stories of improvements provided to the 


The CFAs also need to create a clearer definition of what they mean by a partnership 

or network organisation, as in some cases organisations evaluated under this category 
were not in fact partnerships or network, but rather fairly classic operational NGOs. 
These provide an efficient conduit of CFA funding to community level and are therefore 
important in their own right, but there needs to be greater clarity on the mandates and 
functions of these organisations, and levels of representation and participation within 
them, in terms of beneficiaries and stakeholders. 


Cordaid and ICCO are also not yet creating effective links between their partners and 
relevant networks in the countries they work in, nor are they playing a role in bringing 
together like-minded organisations within countries, which has the potential for creating 
greater synergies. Plan is also not creating sufficient links between its various project 
teams, nor between these teams and local/national networks that could allow them to 
scale up innovative practice beyond the scope of their projects. This has resulted in some 
‘atomisation’ of programmes within countries, with little sense of an overall, coherent 
health strategy that works towards improved population health. 


Recommendations for improving partnerships/networks 


Cordaid and ICCO 

e Networking and advocacy partners need particular support in developing ways of 
measuring the outcomes of their less tangible activities. The strength and results 
of partnerships can be assessed within the partnerships themselves, using validated 
tools, one of which was introduced in this evaluation. Cordaid and ICCO should 
also support advocacy partners to develop key indicators that will help them to 
benchmark progress towards reaching their advocacy objectives. 

¢ Once partnerships and networks have undertaken a more robust assessment of their 
partnerships, Cordaid and ICCO should support these organisations to build on their 
members’ strengths so that they become more influential. At present, many of the 
partnerships and networks supported are not creating the synergistic effect expected 
from organisations working together. ea 

© Cordaid and ICCO already provide some South-South networking opportunities for 
partners and these should be continued and expanded to include a cross section of 
both organisations’ partners where appropriate. Innovative work done by partners 
on gender-based violence should be a key area of expansion as some of the 
organisations working in this area remain small and isolated. 

© Inthe DRC, Cordaid and ICCO should work together with their faith-based partners 
and other donors to develop a national faith-based health association that can 
advocate on behalf of faith-based health service providers as well as provide a 
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s. It may be helpful to involve 


uality assurance of health service si ange 
framework for'qualty mbrella organisation in Zambia. 


CHAZ, to draw on its experience as an effective u 


9.4 Summary of value added by CFAs to partner programmes 


The CFAs are bringing added value to the work of their partners, primarily through 
flexible and consistent funding and support for capacity building. While there is much 
overlap between the policies of the CFAs, each also has unique elements to its approach. 
Assessment is limited by the lack of identified added value in CFA strategy, in terms of, 
for example capacity building and human resource development, global North-South 
advocacy, promoting innovation and facilitating linkages and information exchange. 


Examples of added value include Cordaid’s support to strengthening BDOMs and 
provider partners in Zambia in building up their managerial competencies, which is 
contributing to making the organisations more efficient and effective, and broadening 
their funding base. Respondents in DRC felt that the presence of a local office for 
Cordaid facilitates interaction between Cordaid and its partners, enhances their 
participation in donor coordination, and facilitates advocacy for improved legislation 
and policy making at the national level. Cordaid’s technical assistants are also filling 
in human resource gaps in hospitals and clinics where it is very difficult to recruit and 
retain staff, while Cordaid is also providing packages of training and incentives to 
national staff. 


ICCO has a very flexible approach to its funding, allowing it to build partnerships with 
civil society institutions, while also maintaining links into faith-based organisations, 
which are critical health service providers in many countries. ICCO’s approach 

opens opportunities to small organisations that are usually below the international 
development radar. An observed significant added value of ICCO’s support is that it 
enabled partners to adopt and maintain a more holistic approach, which does not 
address health in isolation but as an integral aspect of development. It was felt that 

the importance given to community capacity building was very valuable though 
extremely challenging in war-stricken areas. Support rendered by ICCO is strengthening 
civil society in Eastern Congo in terms of providing specialised services as with CME, 
increasing training capacity for community health and creating enabling environments 
(IPASC), and networking with grassroots organisations (BOAD and ELIMU). 


Plan’s field work in each country brings a more ‘hands on’ approach to community 
(and civil society) capacity building through its community facilitators and through 
facilitating training for health facility staff. Plan Netherland’s contribution is to provide 
active support for this work, and to promote the child-centred approach. 


Numerous areas of effective innovation were also identified by this evaluation, 

for example, in Plan’s programmes for early childhood care, in the organisations 
supporting women who experienced sexual violence (Cordaid-DRC and ICCO-Nepal), in 
institutionalising new approaches with government and influencing policies on urban 
primary health care (ICCO-Nepal), AIDS outreach and care in Zambia (Cordaid) and 
performance-based contracting in the DRC (Cordaid). 


Some of the partner organisations visited have been involved in a number of capacity 
building workshops in Zambia and DRC. ICCO has also organised regional-level sharing 
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delivery partners had taken place in Nepal. N i 
among all NGOs. While the conflict situation 
this evaluation was able to conduct two work 
particularly tense periods shows that it is pos 


may explain some of this, the fact that 
shops with health-related partners during 
sible to facilitate greater networking. 


While many positive results of the CFA - southern organisation partnership were cited 


those partners that took part in the evaluation also identified many areas where this 
relationship could be improved. These included: 


e High turnover of CFA programme officers has disrupted communications and 
complicated relationships with partner organisations. 

e Rapid changes in policy direction, with little consultation and explanation has led to 
a great deal of confusion amongst partners. 

e Little support to organisational development activities has made it hard for partners 
to respond to requests for strategic plans and other CFA demands. 

e Weak engagement in the advocacy work of southern organisations wastes an 
opportunity for linking southern and northern advocacy networks. 


This evaluation has found that added value could be strengthened. The CFA 
organisations have undergone a period of change in the last few years, which has also 
brought with it some significant changes in policy and strategy. All CFAs are struggling 
to some extent to see these changes implemented in concrete programme terms. Where 
partners are already well funded by other donors (e.g. the majority of service provider 
organisations evaluated) Cordaid and ICCO in particular should reconsider how their 
funding can best be used to ensure that their more progressive policies in gender and 
rights-based approaches can be implemented through partners. 


Recommendations for increasing added value of CFAs 


All CFAs 

e As an initial step CFAs should support cross-learning opportunities that bring 
together partners in single countries or regionally to share experiences and discuss 
good practice. These could be done at a sectoral level, though much of the good 
practice observed during the evaluation has cross-sectoral application. This should 
include making more deliberate efforts to link partners involved in running networks 
for advocacy-related work and service delivery partners. ; 

¢ CFAs should also develop strategies to strengthen the northern voice of their 
southern partners, working with partners to agree on priority issues for advocacy 
strategies developed by CFA policy units. 


Cordaid and ICCO ; 

° Both Cordaid and ICCO need to elaborate more explicit links, through strategic 
planning processes, between their global policies related to access to health and basic 
services and the programmes they support in countries. These links should delineate 
how the CFAs are adding value to partners programmes, including: 

- Challenging prevailing social and organisational norms relating to gender, - well 
as paternalistic attitudes of some partners towards communities they work with 
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sms for cross-learning between partner organisations to 
nnovation and to promote stronger coherence. 

t in DRC and Nepal where normal coordination 

d need developing. This could start with 
ion mechanism would best suit 


- Developing mechani 
ensure the spread of good i 
This is particularly importan 
mechanisms are weak or non-existent an 
an exploration with partners as to what coordinat 
their needs and could extend to non-CFA partners in time. 

e CFAs should strive for greater stability of programme officers to allow for more 
consistent and longer-term contacts between partner organisations and the relevant 
CFA. Where this is unlikely, or hard to influence, the CFAs need to develop a more 
consistent communication strategy with partners, which allows them to engage more 


in dialogue with the CFA organisation concerned. 


Plan 
e Plan should encourage greater cross-learning between staff working on different 


projects, as innovations are not being spread effectively across each country’s 
programme. 

e Plan staff should be provided with further training and support to challenge social 
norms within the CSOs they work with that exclude particular groups, and to use 
their prominent role in the communities they work in to advocate more directly for 
the rights and needs of the most vulnerable. 
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Assessment of the Final Report Co-financing 
Agencies Evaluation of Health Theme 


MBN Evaluation Reference Group 


The Evaluation Reference Group (ERG) has been ass 
‘quality of process and results’ of the CFP evaluatio 
comments on the initial Terms of Reference for the 
report submitted in January 2005. With respect to t 
between, findings, analysis, conclusions and recommendations. In this assessment report 
we focus on the results of the evaluation as reflected in the final report ‘Co-financing 
Agencies Evaluation of Health Theme’ (submitted October 2005). 


igned the task of assessing the 

n studies. The ERG has given 
Health evaluation and on the draft 
he latter, the ERG pointed to gaps 


We are aware that there have been operational difficulties in this evaluation, due to 
the difficulty in finding a good evaluator. Also the evaluators decided to concentrate 
on two relatively difficult countries for field-visits: Nepal and the Democratic Republic 
of Congo. It is remarkable that despite the instability in the national contexts, the 
evaluations were completed within the time-frames set and with good results. 


The ERG considers the final report to be an interesting, and balanced assessment of 
the contributions of co-financing agencies and their country level partners in terms 
of improving access to health-related services. The conclusions of the assessment are 
well supported by evidence given in the report. The report is moreover well written 
and provides many clear lessons for the co-financing agencies. The ERG is particularly 
pleased with the choice of countries - including Nepal and DRC, which are countries 
with a relatively weak state. It is in such settings that NGOs have much ‘added value’. 
The description of the national contexts is furthermore well done. 


The ERG notes that the evaluators could have compared the different visions, and 
interventions strategies of the three CFAs (ICCO, Plan and Cordaid) more systematically. 
The evaluators appear impressed by the effectiveness of Plan interventions, and are 
concerned about the ‘flexibility’ in the approach of ICCO. However, from the perspective 
of the partner-organizations the added value of the flexible approach of ICCO could be a 
higher degree of local ownership of the programs funded by ICCO. More reflection on the 
relative strengths and weaknesses of the three distinct modes of support is needed. Such 
an elaboration is missing, and could provide good opportunities for learning across the 
CFAs in the Netherlands as well as in the developing country contexts. 


The ERG is surprised and concerned that in settings where church-based health 
programs have been providing health services for decades, the evaluators were nan 
constrained by lack of data on impact of these services. This points to weak monitoring 
and evaluation mechanisms that need to be addressed as a matter of urgency, as 
recommended by the evaluators. 


The ERG is concerned about the inequitable access to basic services caused by user- 
fees, given that exemption mechanisms generally do not work well. The ERG points 
to the need to consider other, cost-sharing models for health financing. Cost-sharing 
mechanisms exist traditionally in many societies. The evaluators have not taken such 
other models into considerations. The evaluators have a mixed assessment of the 
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fees for basic health services. On the one hand the evaluators 
o the use of services, however, they also 


h needed resource generation. 


appropriateness of user- 
acknowledge that user fees can be a deterrent t 
value positively that user fees contribute to muc 


The evaluators are positive about the quality of the health services provided, but the 
methods used to evaluate quality are limited. The ERG comments that technical quality 
of the services was not assessed by the evaluators. The quality assessment refers to 
perceived quality, from the perspective of beneficiaries. 


The ERG is concerned about the finding that many of the Catholic health services, 
funded by Cordaid, will not engage in the delivery of sexual and reproductive health 
care. This is a serious problem, especially in the African countries covered by the 
evaluation, which are highly affected by the AID epidemic. In DRC the church-based 
institutions are the main source of basic health care for entire populations. Clearly there 
is a need for advocacy to broaden the scope of services offered. 


Finally, the ERG discussed the sustainability of the institutions providing basic 

health services and their linkages with local government. Many of the church-based 
institutions have been receiving external support from CFAs for decades. In countries 
like Zambia, where health reforms are being implemented, the NGO health care providers 
need to rethink their relationship with local government. The evaluators point out that 
the relationship between the two partners is not yet based on trust and transparency. 
This point clearly requires attention from the CFA staff, included on the value added by 
NGO basic health provision. 


The report provides important lessons, which can guide partner organizations and CFAs 
in future policy making. The recommendations are also well structured in line with the 
main themes of the evaluation. They provide specific suggestions for each of the CFAs 
on improvements in 

e health related visions, strategies and activities 

¢ quantitative and qualitative impact of health-related services 

¢ partnerships and networking 

e CFA value added. 


The recommendations for each of these themes are clear, specific, and well supported by 

evidence. The ERG would like to highlight the following five specific recommendations, 

which are highly relevant and directed at all CFAs 

1 All the CFAs need to reconsider their own and partner strategies that depend on user 
charges for services. 

2 New systems for monitoring health impact of the intervention strategies need to be 
developed by the CFAs. 

3 More support to partners is needed, in order to understand and integrate gender and 
rights based approaches in practice (this point is especially a challenge for CORDAID) 

4 All CFAs should continue to focus on the strengthening of community based 
programs, as they have the largest effect on the health of the poor. 

5 CFAs should support learning opportunities among their partners in single countries, 
or regionally. 


The ERG is pleased that all three evaluated CFAs have discussed the final report of the 


evaluators and are in the process of incorporating the lessons learnt into their health 
policies. 
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Appendix 2 
Terms of Reference 


Evaluation of the Netherlands Co-Financin 


g Programme Heal 
Plan Netherlands and Cordaid TORs/Opera caith Programme of ICCO, 


tional Methodology 


Background 


This TOR is for the joint evaluation of selected health (and related) programmes 
supported by three Dutch Co-Financing Agencies (CFAs): Cordaid, ICCO and Plan 
Netherlands. The health programmes are managed with partner organisations in three 
countries selected by the CFAs (Nepal, Zambia and the Democratic Republic of Congo). 
The evaluation was commissioned by Cordaid, PLAN Netherlands and ICCO, the latter 
of which is taking responsibility for coordinating the health component evaluation. It 
will contribute to the wider, ongoing Co-Financing Programme evaluation (2003-2006), 
which includes studies undertaken on six additional themes. More detailed background 
information explaining the rationale behind this evaluation can be found in the 
‘Background Paper’ in Appendix 1. 


Cordaid, ICCO and Plan Netherlands decided to commission a programme evaluation to 
gain better insight in the results of their support of health-related activities for poverty 
reduction and to learn from their practice. The evaluation is seen in relation to the 
ongoing evaluation efforts related to the CFP (Co-Funding Programme) that have been 
undertaken since 1980. Based on reviewing the results of previous evaluations the three 
organisations decided the present evaluation must give attention to both direct service 
delivery and to civil society building aspects of the CFA-supported interventions in the 
health sector. 


This document outlines the aims, objectives, framework and methodology for the 
evaluation of ICCO, Plan Netherlands and Cordaid’s health programmes proposed by 

the evaluation team. This Master TOR should be read in conjunction with the Technical 
Proposal submitted by the Institute for Health Sector Development (IHSD) ( Appendix 2). 
The Technical Proposal provides information on IHSD’s qualifications for undertaking 
this work, the general framework of the evaluation and the CVs of the two international 
team members. 


Evaluation aim and questions 


The aim of this evaluation is to gain insight into the contributions of the CFAs and 
their partner organisations to the improvement of the sustainable use of, access to and 
control over health-related services for the poorer sections of the population, with a 
special emphasis on women and youth. 


Questions to be answered by this evaluation are provided below, with sub-questions/ 
areas of enquiry: 
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ties do partner organisations have to improve 


isions, strategies and activi 
Sue : Ith services by potential users? 


access to, use of and control over basic hea 


alitative and quantitative results of these programmes and for whom 


the qu : 
2. Witiat are the 4 use of and control over basic health (-related) 


do they improve sustainable access to, 
services? 


3 To what extent do complementary partnerships, participation in networks and/or 
alliances contribute to better performance of partners? 


4 What are the demonstrable results or outcomes from partnership working in terms of 
performance of partner organisations and impact on the health system in general? 


5 What is the added value of the CFAs’ support for the partner organisations and their 
programmes? 


Analytical framework 


The above questions will be answered using an analytical framework that includes seven 
variables that the evaluation team considers to be the core to the questions that need 
answering. As stated in the Technical Proposal, ‘Programme outcomes are delivered or 
mediated through a number of processes or outputs, including direct service provision 
to poor communities, civil society building through development of alliances and 
networks, citizen and client voice and policy-advocacy activities, often undertaken by 
the intermediary partner organisations. Investments/inputs to these processes by the 
CFAs are the key strategies of a) working in partnership with a number of groups in 
each country and b) facilitation of organisational development or capacity building’. As 
this evaluation seeks to understand both outcomes and processes, the variables to be 
measured (either quantitative or qualitative) are as follows: 


Impact/effectiveness: | Achievement or likely achievement of objectives 
Impact/influence of activities on other local and/or 
national bodies 


Quality: Technical quality of interventions 


Coherence: With national health policy and strategy 
With Co-Financing Programme health policy and strategy 
With individual CFA policy and strategies 
With Dutch government policy and strategies 


Participation: Of partner organisations in planning and monitoring of projects 
or interventions 
Of beneficiaries in planning, management and monitoring/ 
evaluation of projects or interventions 
Of women, young people and socially excluded in the planning, 
management and monitoring/evaluation of projects or 
interventions. 
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Sustainability: 


Added value: 
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Of the expected Outputs and outcom 
intervention 


Of the partner Organisations themselves 


es of the project or 


Benefit to individual] Organisation of partnership/networks 
Complimentarity of members of partnerships/networks 
Effectiveness of Partnership working or networking for 
achieving intended objectives 


Of civil society interventions to overall government public 
health activities 

Of partner activities to improvements in service delivery and 
improvements in policy (scaling up) 

To partners/projects of the relationship with the CFA 


As can be seen from the above, there will be multiple levels of analysis within this 


evaluation, including beneficiaries, partner organisations, civil society organisations, the 
CFA organisations and the Co-Financing Programme. 


The following tables provide a schema linking the evaluation questions with this 
analytical framework and the evidence to be collected. The first table (pages 3 and 4) 
is oriented towards direct service delivery partners. The second table (pages 5 and 6) is 
oriented to those partners more involved in advocacy and networking. 
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Sources of evidence/data 


e Partner organisation strategic 
plans and programme documents 

e Interviews with partner 
organisation or project staff 

e Interviews/focus group discussion 
with beneficiaries 


Table 14 - Evaluation Questions 
Related Variables in Analytical Framework 


1a_Impact/effectiveness - Have partner organisations 
achieved, or are they likely to achieve, their 
organisational objectives? eg 

1b Participation - To what degree have organisational 
members and beneficiaries contributed to developing the 
vision, strategies and objectives of the partner? 

1c Coherence - Is the vision of the partner organisation 
coherent with government policy (in terms of health and 
of community development?) 

1d Sustainability - To what degree are the partner 
organisations’ visions, strategies and activities 
sustainable? 

1e Added value - what has been the contribution of 

relationship with CFAs to the development of partner 

organisations’ vision, strategies and activities? 


Impact/effectiveness - Has the partner/project achieved, 

or are they likely to achieve, their expected health- 

related objectives? What is the impact on different 

vulnerable groups, especially women and youth? 

2b Quality - Are the expected objectives technically valid 
given the context? Is the quality of care provided of 
good standard? 

2c Coherence - Do the health services and activities 
provided fit within the country's health policy? Are they 
coherent with CFA, CFP and Dutch government policies? 
Are there innovative aspects that could influence 
improvements in service delivery locally and nationally? 

2d Participation - To what degree have beneficiaries 
participated in decisions on what health services 
and activities are provided? To what degree do they 
participate in the management and monitoring of health 
services and activities? How have gender aspects been 
handled? 

2e Sustainability - How sustainable (financially and 
institutionally) are the health interventions provided? 

2f Value - added - Have other organisations (local or 

national) adopted programme strategies? 


1 Which visions, 
strategies and 
activities do partner 
organisations have to 
improve access to, use 
of and control over 
basic health services by 
potential users? 


Project or health service data 
on health indicators specific to 
programme interventions, on 
number of beneficiaries and cost- 
effectiveness; 

¢ Observation of service delivery 
in health centres, if related to 
programme; 

¢ Comparison of programme 
proposal with national health 
policy documents, CFA, CFP 
and Dutch government policy 
documents; 

e Interview with organisation staff 

¢ Interviews with health service 
staff 

Interviews with beneficiaries 

Interviews with communities in 

programme catchment area 


What are the 
qualitative and 
quantitative results 

of these programmes 
and for whom do they 
improve sustainable 
access to, use of and 
control over basic 
health (-related) 
services? 


Questionnaire/interviews with 
organisation staff; 

e Interviews with other members of 
partnerships; 


3 To which extent 
do complementary 
partnerships, 
participation in 
networks and/or 
alliances contribute to 
better performance of 

partners? 


Impact/effectiveness: How are the partnerships of 
partner organisations functioning and what do they 
contribute to a better performance of the individual 
organisations? 

3b Coherence - Are the partnership and network 
relationships coherent with CFP policy of how they 
should work? 

3c Participation - what is the level of involvement of 
different partners in the relevant partnerships? 

3d Sustainability - Are the relationships and work 
established by the partnerships sustainable? 

3e Added value - What has been the extent of the role of 

CFAs with the partnerships or networks that partner 

organisations have been involved in? How is that role 

appreciated by partners? 


partnerships, 


of the partner 


general? 


support for th 


programmes? 


With what results 
do (complementary) 


participation in 
networks and/or 
alliances contribute to 
a better performance 


organisations and 
the health system in 


5 What is the added 
value, in terms of the 
results and partnership 
aspects, of the CFAs 


Organisations and their 


ApPenvix 2 - Terms oF REFERENCE 


objectives? 


results? 


partners? 


e partner 


strategy? 


Impact/effectiveness - What has been the contribution 
of membership in the partnershi 


the partner Organisation/project 


Impact/effectiveness - What has been the contribution of the 
CFA to the achievement or expected achievement of partner 
organisation results? 

5b Quality - To what extent has support from the CFA improved 
the technical and organisational quality of the health (- 
related) programme? 

5c Coherence - To what degree are the values and strategies of 
the CFA coherent with those of the partner organisation? 

5d Participation - To what extent do partner organisations 

influence or participate in decisions about CFA policy and 


P Or network in helping 
to achieve its programme 


40 Quality: Are the interventions undertaken or advocated by the 
partnerships and/or their members of good technical quality? 

4c Coherence: Are the interventions undertaken or advocated by 
the partnership, network and/or their members coherent with 
national, as well as CFA and CFP policies? 

4d Participation - what is the level of involvement of different 
partners in determining the aims and objectives of the 
partnership or network? 

4e Sustainability - How sustainable are the contributions of 
the partnerships to helping individual members achieve their 


4f. Added value - What has been the extent of the role of CFAs 
with the partnerships or networks that partner organisations 
have been involved in? How is that role appreciated by 


5e Sustainability - How sustainable is the relationship between 
the CFA and partner organisations? 
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* Questionnaire/interviews 
with organisation staff; 

¢ Interviews with other 
members of partnerships; 


Interviews with 
Organisation staff 

¢ Interviews with other key 
stakeholders at national 
and local level 

¢ Comparison with 

other donors (provided 

comparable data are 

available) 
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Sources of evidence/data 


e Partner organisation 
strategic plans and 
programme documents 

¢ Interviews with partner 
organisation or project staff 

e Interviews/focus group 
discussion with beneficiaries 


on Questions 
Related Variables in Analytical Framework 


1a _Impact/effectiveness - Have partner organisations achieved, 
or are they likely to achieve, their organisational objectives? 

1b Participation - To what degree have organisational members 
and beneficiaries contributed to developing the vision, 
strategies and objectives of the partner? How have gender 
aspects been handled? 

1c Coherence - Is the vision of the partner organisation 
coherent with government policy (in terms of health and of 
community development?) 

1d Sustainability - To what degree are the partner 
organisations’ visions, strategies and activities sustainable? 

1e Added value - what has been the contribution of relationship 

with CFAs to the development of partner organisations’ 

vision, strategies and activities? 


Impact/effectiveness - Has the partner/project achieved, 

or are they likely to achieve, their expected advocacy and 

networking related objectives? 

2b Coherence - How do advocacy and networking strategies 
sit with similar work by other organisations? Have they 
contributed to national policies re: poverty reduction? What 
innovations have they introduced that have been adopted 
by higher level decision makers? Are activities coherent with 
CFA, CFP and Dutch government strategies? 

2c Participation - To what degree have beneficiaries 
participated in decisions on what the advocacy messages 
should be? To what degree do they participate in influencing 
and networking activities? 

2d Sustainability - How sustainable (financially and 
institutionally) are the concerns that form the basis of 
advocacy? 

2e Added value - How have CFAs contributed to the 

achievement of networking-related objectives? Has 

networking/advocacy contributed to scaling up of activities 

at other levels in the system? 


Table 2 - Evaluati 


1 Which visions, 
strategies and 

activities do partner 
organisations have to 
improve access to, use 
of and control over 
basic health services by 
potential users? 


Project data on who has 
been the target of advocacy 
and messages sent; 

e Interviews with advocacy 
targets where possible, on 
influence of networks and 
campaigns; 

¢ Comparison of programme 
proposal with national health 
policy documents, CFA, CFP 
and Dutch government 
policy documents; 

e Interview with organisation 
staff 

Interviews with beneficiaries 

Interviews with communities 

in programme catchment 

area 


What are the 
qualitative and 
quantitative results 

of these programmes 
and for whom do they 
improve sustainable 
access to, use of and 
control over basic 
health (-related) 
services? 


Questionnaire/interviews 
with organisation staff; 
e Interviews with other 
members of partnerships; 


3 To which extent 
do complementary 
partnerships, 
participation in 
networks and/or 
alliances contribute to 
better performance of 

partners? 


Impact/effectiveness - How are the partnerships of partner 
organisations functioning and what do they contribute to a 
better performance of the individual organisations? 

3b Coherence - Are the partnership and network relationships 
coherent with CFP policy of how they should work? Have 
they influenced CFP policy in turn? 

3c Participation - what is the level of involvement of different 
members in the partnership? To what degree are women and 
youth included? 

3d Sustainability - Are the relationships and work established by 
the partnerships sustainable? 

3e Added value - What has been the extent of the role of CFAs 
with the partnerships or networks that partner organisations 

have been involved in? How is that role appreciated by 

partners? 
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4 With what results 
do (complementary) 
partnerships, 
participation in 
networks and/or 
alliances contribute to 
a better performance 
of the partner 
organisations and 
the health system in 
general? 


4a Impact/effectiveness - What has been the contribution 
of the partnership or network in helping the member 
partner Organisations/ projects to achieve their programme 
objectives? To what degree has participation contributed to 
scaling up of activities to higher levels? 

4b Quality - Are the interventions undertaken or advocated 
by the partnerships and/or their members of good technical 
quality (in terms of communication strategy)? 

4c Coherence - Are the interventions undertaken or advocated 
by the partnership, network and/or their members coherent 
with national, as well as CFA and CFP policies? What has been 
the contribution of the network Organisation to the country 
policy of the specific organisation and its partners? 

4d. Participation - what is the level of involvement of different 
members within the partnership in determining the aims and 
objectives of the partnership, network and areas to influence? 

4e Sustainability - How sustainable are the contributions are 
the inputs provided by the umbrella organisation to its 
individual members? 

4f Added value - What has been the extent of the role of CFAs 
with the partnerships or networks that partner Organisations 

have been involved in? How is that role appreciated by 

partners? 


* Questionnaire/interviews 
with organisation staff; 

* Interviews with other 
members of partnerships;. 


5 What is the added 
value, in terms of the 
results and partnership 
aspects, of the CFAs 
support for the 

partner-organisations 

and their programmes? 


Impact/effectiveness - What has been the contribution of 
the CFA to the achievement or expected achievement of 
partner organisation results? 

56 Quality - To what extent has support from the CFA improved 
the technical and organisational quality of the health (- 
related) programme? 

5c Coherence - To what degree are the values and strategies of 
the CFA coherent with those of the partner organisation? 

5d Participation - To what extent do partner organisations 
influence or participate in decisions about CFA policy and 
strategy? 

5e Sustainability - How sustainable is the relationship between 

the CFA and partner organisations? 


Interviews with organisation 
staff 

¢ Interviews with other key 
stakeholders at national and 
local level 

¢ Comparison with other 

donors (provided comparable 

data are available) 


Note: Measuring the impact of networks, advocacy and policy work (policy-advocacy-evidence loop) will be done through: 


Looking at the overall aim or objective of the network or advocacy work and its evidence base ad 

e First order change (outputs) e.g. number of groups formed as a result of activities, or number and type of communication 
messages on area to be influenced (advocacy) i 

e Second order change (effects) e.g. evolution of groups through membership, creating community infrastructure or changes 
in community activities. For policy, changes in opinions generated by advocacy amongst key decision makers i 

© Third order change (impact) e.g. changes occurring in community or organisation norms, degree of self-reliance. For policy 
work, whether changes in policy have occurred, and whether policy change has been implemented 


These levels of change will be ascertained through the questions asked within the framework. 


. 


Outputs 


1 Country report from each of the three countries that provides detail for each CFA 
programme separately, identifies similarities and differences, and provides analysis o 
what is working well and less well. - 

2 Aconcise synthesis report (60 pages) drawing together the results and analysis from 

all three country studies, as well as analysis of organisational policy issues from each 
of the three CFAs, resulting in recommendations on areas for improvement to partner 


organisations, the CFAs and the Co-Financing Programme. 
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Methodology 
Consultant review teams 


Overview work and synthesis 


IHSD Team Leader 
International consultant 


Individual country studies 
Organisation development specialist with participatory evaluation skills 


Public health specialist/demographer 
Poverty and gender specialist with participatory evaluation skills 


The specific work functions of each team member are detailed in separate TORs in 
Appendix 3. IHSD Team Leader and International Consultant CVs can be found at the 
end of the Technical Proposal in Appendix 2. CVs for the Country Team Leaders can be 


found in Appendix 4. 


The methodology is based on well-tested case study methods used in similar styles of 
research. Both Marsden et al. (1994) and Roche (1999) either reviewed or used the case 
study approach as a means of evaluating the impact of development assistance. The 
methodology in this study is based on the learning and best practice advice outlined by 
both groups of authors. 


The evaluation will consider the CFAs’ programmes in three different countries; 
Democratic Republic of Congo, Nepal and Zambia. The three CFAs formulated the 
following criteria for the selection of the case study countries: 


e Substantial financial CFA contribution to health-related programmes 

e¢ Considerable number of partner organisations active in the health (-related) sector 

e Existence of a network, support organisation or other formal or informal structure in 
the health system in which the partner organisations participate in order to undertake 
lobby, advocacy or other activities or receive support from it 

e Evenly spread over the continents 

e Haven’t had too many evaluations in the past few years 


Specific partners to be involved in the study will be selected according to the following 
criteria: 


Aims and objectives of the partner organisation and/or the health interventions 
Level of activity of the partner in the health field 

Balancing rural and urban interventions 

Mixture of different types of organisations and interventions (direct service delivery, 
advocacy, community health promotion and empowerment etc.) 


Throughout the evaluation gender aspects will be taken into account. 
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Process 


1 Recruit teams and organise key dates and deadlines for the review process 


2 IHSD team leader and international consultant will 
organisations to agree on study methodology and t 


meet with three Dutch 
imetable. 


3 Consultation and review process to include: 


Review of organisation policy and pro 
Table 1 and Table [#?] in Appendix 5 
Review of project and partner details to select which 
be used in this evaluation 

Interviews with ICCO, Cordaid and Plan Netherlands staff to collect information 
on outputs to date, achievements to date and organisational as well as partnership 
aspects of work 

Inception workshop in each country, with at least one of the international 
consultant team, country research team members and selected partner or project 
staff to review and revise methodology for study to ensure it fits with country 
context, as well as to identify documents, data and stakeholders to be reviewed/ 
interviewed as part of the study; Workshop will also include partnership analysis 
work 

Document review of individual partner or project documentation to identify 
organisational development aspects 

Interviews with health service staff to understand trends in organisational 
development, service use, achievements and areas for improvement 

Data analysis of programme data and comparison data to identify trends, 
differences and any effects attributable to programme interventions 

Interviews with key health, NGO and other appropriate stakeholders to include 
questions on degree to which they think programmes have met their outputs and 
intended impact: What are the gaps? What could have been done differently or 
better? 

Observation of health service delivery in selected health centres that are targeted 
by programme 

Focus group discussions with beneficiaries (especially women and young people) 
to find out their views of services provided, the organisations providing the 
services, level of user involvement, who is included/excluded, achievements and 
areas for improvement. 


gramme documentation against questions in 


organisations or projects will 


4 Write up findings of each country review as per expected outputs. 


5 Hold workshop with’partner and project staff, as well as appropriate stakeholders in 
each country to present and discuss results and analysis. 


6 Write up draft synthesis report pulling results and analysis together, to be shared 
with and commented on by each country team. 


7 Hold final meeting with CFAs to discuss findings and reports. 


8 Finalise and submit report. 
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Proposed time frame for work 
fa __ emp ates eae [Timing ___—_—_| No. of Days 
Reems 


Recruit teams ; 
Review CFA health programme and Janneke Roos and Cindy Carlson 10 person days 
policy documents 


Review team coordinator Early September One person day for each 
and other international team international consultant 
member[WAAROM HIER TITELS 
EN VERDER DE NAMEN?] 

Inception workshop in each country | Cindy Carlson, Janneke Roos and | Zambia - September 5 person days each for 


and initial stakeholder interviews relevant country team international 
DRC and Nepal- 


October Up to 2 person days for 
each country team 


Data collection - health statistics Country research team members | September-November | Up to 30 person days for 
and interviews each in-country team 
Write-up country findings Country research team By end November Up to 6 person days for 
each in-country team 
Partners Workshops Country research teams and End November/early Up to 2 person-days for 
international consultant December each country team 


Write up final country and draft Cindy Carlson and Janneke Roos | December Up to 18 person days for 

synthesis reports international consultants 
Up to 2 person days for 
each country team 


Workshop in the Netherlands with | Cindy Carlson Mid to end December _ | Up to 2 days 
CFAs 


Note: Division of days within each country team will be dependent on expertise and role each team member will play. This will 
be determined after the in-country inception workshop in discussion with the relevant international consultant. 


Review team meeting with CFA 


References 


Marsden D, Oakley P and Pratt B (1994) Measuring the Process: Guidelines for Evaluating Social 
Development. Oxford: Intrac 

Roche C. (1999) Impact Assessment for Development Agencies: Learning to Value Change. 
Oxford: Oxfam Publication 


Appendix 3 
Technical Proposal 


Evaluation of The Netherlands Co-Financin 
g Progr 
Plan Netherlands and Cordaid gramme Health Programmes of ICCO, 


Technical Proposal (ref BK/Onderzoek/MvdG/04.0034) 


Presented by: IHSD Ltd, Institute for Health Sector Development 
June 2004 


Introduction and policy context 


IHSD experience and competency 

This proposal is for the joint evaluation of selected health (and related) programmes 
supported by three Dutch CFAs (Co-Financing Agencies, Cordaid, ICCO and Plan 
Netherlands). The health programmes are managed with partner organisations in three 
countries selected by the CFAs (Nepal, Zambia and the DR Congo). The evaluation 

is commissioned by ICCO, which is taking responsibility for coordinating the health 
component evaluation. It will contribute to the wider, ongoing Co-Financing Programme 
evaluation (2003-2006), which includes studies undertaken by other CFAs on six 
additional themes. 


The proposal is from the Institute for Health Sector Development (IHSD Ltd, UK). IHSD 
is pleased to bid for this evaluation as an organisation with extensive experience in the 
health sector (with public, private and civil society players). 


IHSD is specialised in health and HIV/AIDS work in low and middle-income countries. 
With its sister company HLSP Consulting, country level experience includes short term 
technical assistance for a range of bi- and multilaterals, and NGOs in health sector 
development, including extensive monitoring and evaluation activities. The organisation 
has substantial knowledge of current issues in aid policy and delivery, challenges for 
national governments and civil society, and the international and national operating 
contexts facing bilateral agencies and their NGO partners in the development process. 


In particular, IHSD has held a series of contracts with the UK Department for Inter- 
national Department (DFID) for ten years to provide policy and consultancy support to 
central and country-focused work. DFID’s support at country level includes both civil 
society development and public sector support. 


Recent successful bids for long-term project management include provision of TA to 
Zambia's National AIDS Commission and the multisectoral response to HIV/AIDS and to 
the safe motherhood programme in Nepal, both of which involve significant components 


for civil society strengthening activities. 


IHSD has an established track record in monitoring and evaluation, survey design, pe 
policy analysis. Commissions include mid-term reviews, output-to-purpose reviews an 


final programme reviews and evaluations. 
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Recent assignments include: 


and Reproductive Health Programme in Malawi 


f the National Sexual 
aie rformance of the African Youth Alliance Programme 


e A mid-term assessment of the pe 
(AYA) in Botswana, Ghana, Tanzania and Uganda 7 

© Review of the Community Health Financing Project in Uganda, including 
recommendations on how to support local organisations, mainly not-for profit 
hospitals, to develop local health insurance arrangements in order to improve access 
to health services for the residents in their area 

© Mid term review of the 1999-2002 strategy and plan of action of TASO, a major NGO 
providing support for people living with, and affected by AIDS in Uganda 

© Global Stop TB Partnership evaluation to examine the functions and structure of the 
Partnership and its components, and the degree to which the operational model of 
the partnership fosters progress at country level. 

e IPPPH Study (Initiative on Public-Private Partnerships for Health Global Forum for 
Health Research) to assess the health and health systems impact of public-private 
partnerships for improving access to pharmaceuticals, in Uganda, Zambia, Sri Lanka 


and Botswana. 


IHSD is also involved in both training and policy review work on broader aid 
management and policy issues, including PRSPs, Sector Wide Approaches (SWAps), 
budget support, and the new Global Health Partnerships such as the Global Fund to 
Fight AIDS TB and Malaria. (See annex B for a brief description and relevant experience 
of IHSD). 


International development policy context 

The major international trends are reflected in The Netherlands overall strategy for 
development policy. This shapes the CFP, and therefore has an influence on CFA 
objectives and processes for supporting effective partnerships and programmes at 
country level. CFAs also have an important role in their own right in North-South 
networking and support for policy advocacy, and in the development of independent 
civil society voices. In reviewing the performance of CFAs and their country level 
partners, it will be important to bear these sometimes conflicting issues and objectives in 
mind and consider both whether the programme activities achieved what was intended 
(e.g. impact defined in terms of project objectives) as well as their fit with current over- 
arching policy priorities and policy concerns. 


Major trends and issues in the wider context are set out below. 


¢ The move towards longer term, nationally owned development and investment 
strategies for poverty reduction, which encompass poverty reduction and growth 
strategies, sector strategies, and M&E and budgetary frameworks, alongside a drive 
for increased donor harmonisation. Governments are increasingly held accountable 
by their populations through democratic process as well as external partners. 
Policy support is growing for a strengthened civil society role in mechanisms for 
accountability, resource allocation, and priority setting, and for quality bench- 
marking, monitoring and evaluation of service delivery. This environment has 
created great opportunities for development of civil society ‘voice’ and claim-making 
capacity. It can also engender tensions with governmental and financing agencies, 


as Civil society capacity for the ‘challenge function’ strengthens in a maturing 
democracy. 
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providers for scaling up the 


aig W challenges for civil society 
organisations’ role and independence in policy advocacy activities. 


¢ The broader globalising economy is a critical background feature, together with 
international moves towards a rule-based framework for the operation of trade 
in goods and services, and the protection of intellectual property rights. This 
means risks (for the poorest and the disadvantaged in the labour market), but also 
opportunities. For example, a recent assignment for provision of strategic support 
to The Network, a consumer and health rights organisation in Pakistan, built on 
opportunities for North South advocacy and national level networking and policy 
influence to ensure that public health and other interests are protected. 


e The Final Report of the previous evaluation of the Co-Financing Programme (Nov 
2002) outlines several dimensions of structural poverty to be addressed. There is now 
wide consensus on the overall strong association between ill health, especially HIV/ 
AIDS, and poverty, and access to health care as both a human right and a condition 
for growth and poverty reduction. This has a number of implications in terms of 
improving the supply of affordable health care (and related services) that meet 
people’s needs, and increases their demand for, and utilisation of, quality services. 
Financing is particularly critical. Poor households may spend disproportionately 
more of their household income on health than richer ones. Lack of insurance means 
that hospital or emergency care is usually financed through borrowing, often at 
extortionate rates. This can drive the poor into even deeper poverty and is a major 
cause of impoverishment for the not-so-poor. Recent IHSD assignments in Ghana, 
Uganda and Kenya on social and community health insurance schemes highlighted 
the difficulties of establishing schemes where there is limited access to services. 


¢ The rapid growth of private provision in the social sectors such as health, and also 
of the increasingly hybrid nature of the public sector, as staff take on dual roles in 
private and public practice. Much expenditure by households is on private for profit 
providers and pharmaceuticals. A key strategic issue to support interventions that 
ensure poor people get value for their expenditure. Recent work on strategies for 
improving public private sector engagement in nine sub Saharan African countries 
on the future for interventions such as social franchising and contracting indicated 
growing willingness of governments to consider how it, with support from civil 
society, can intervene in improving quality and affordability. 
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Priority issues in selected countries 


ational trends are played out at national level, together with major national 


intern 
Lissiisichauer |, the major challenges (for Government, 


iorities. In the DRC and Nepa 
pl re development partners) are to build the basis for a lasting peace, and 
address the causes of the current conflicts, by supporting much needed economic and 
social transformation. In both countries, human development is central to the PRSP, 
where emphasis is given to increasing the coverage and quality of primary education, 
basic healthcare and other basic needs. However, civil society (and indeed line ministry) 
participation in the PRSP processes have been weak, and the PRSP is widely viewed as a 
‘paper’ policy. 


In Nepal, coordination among the many donors has also been poor, especially as the 
ongoing conflict has shifted the focus from long-term development strategy. However, 
there have been recent positive moves to greater donor harmonisation, since the 
endorsement of the Foreign Aid Policy (FAP) 2002. Special development interventions 
for health, education and local government will be launched in neglected regions. 

The new ministry and donor Health Forum will be tackling health sector development 
and public reform. The health sector does not have a decentralisation strategy, and 
meaningful devolution is a critical reform issue. Currently there are few public 
initiatives with the private sector, although the health sector reform policy is committed 
to developing and implementing a comprehensive public private sector partnership 
strategy. Developing contractual agreements with the private (for profit and not for 
profit) will be explored. 


The conflict is having major impact on health service delivery and staff morale, coupled 
with poor public sector management and lack of measures to address gender and 
ethnicity-related disadvantage. While HIV/AIDS is receiving some attention, maternal 
and neonatal mortality and morbidity, and care for common child illnesses, remain 
major unmet needs. However the MOH has ‘handed over’ the management of over 1,000 
sub health posts to communities to manage (but little management support has been 
given and with no funds for ongoing needs). In this complex environment, there are 
both challenges and opportunities for NGO-supported health programmes to develop 
capacity building processes for community-based management of health services, and 
informed and confident representation in new local and national planning and policy 
fora. 


In Zambia, a stable democracy, HIV/AIDS and its social and economic impact are 
perhaps the major health and development issues that are shaping support offered by 
external partners. The current prevalence rate of 19.7% of people in the productive age 
group (15 to 49 years) has a high economic impact. 


Zambia has a long history of labour mobility that is linked to high transmission rates. 
This situation is exacerbated by the current food crisis, especially in the Southern and 
Western provinces. The agricultural sector supports approximately 50% of households in 
Zambia and is particularly vulnerable to HIV/AIDS. Households affected by AIDS report 
annual income 30-35% lower than unaffected households. HIV/AIDS is also having a 
negative economic impact at macro level. The ILO projects that Zambia will lose 19.9% 
of its labour force (in public and private sectors) by 2020. 
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€ epidemic. Approximately 50% of Zambian 


hospital bed occupancy is HIV-related, and by 2005, AIDS care wil] consume over half 


of Zambia’s recurrent health budget. 


Zambia has developed a multisectoral res 
National AIDS Council and Strategic fra 
Framework places particular emphasis 
to identify then progressively expand best practice, and has identified 42 such 
programmes. Zambia has a vibrant civil society and non-government organisations 
have played a critical role in the national response to HIV/AIDS, although rural areas 
have tended to lack coverage. Over 40% of health care is provided by faith-based 
organisations, and the Churches Medical Association of Zambia (CMAZ) has taken a 
leading role in home cased care initiatives. Prevention responses have been led by a 
range of primarily urban AIDS service organisations. Women’s organisations, such 

as the Young Women’s Christian Association and the Society for Women and AIDS, 
are also active. A second generation of more community-based care responses was 
initiated by the Ndola Catholic Diocese and adopted nationally by the Zambia Catholic 
Secretariat. 


Ponse to HIV/AIDS through setting up a 
mework In 2000. The National Strategic 
On Civil society led catalytic programmes 


Zambia has an impressive commitment to cooperative relations and action. The Zambian 
AIDS response is assisted by a range of multilateral and bilateral actors. Multi-donor 
support for sector-wide approaches in health and education is ensuring a coordinated 
approach to sector funding and strategy. Major issues are the ambitious health sector 
reform programme, with ongoing decentralisation of health service management to 
district level, and strengthening community representation in district level bodies. A 
major issue for civil society and Government is to ensure that the needed focus on 
HIV/AIDS does not compromise access to and quality of other health services. Much 

of the additional finance for HIV/AIDS is being channelled through church and NGO 
networks. A strong civil society response is critical to an effective comprehensive 
response. Ongoing challenges are to build up civil society inputs to governance, policy 
development and monitoring, in addition to increased capacity for scaling up prevention 
and care services. 


Evaluation issues for consideration 

This section of the proposal highlights some of the issues and research questions, 
drawn from and expanding on the evaluation background paper and previous CFP 
reports, which the evaluation framework and methodology will need to address (to be 
developed during the inception phase). In the next sections, the proposal then specifies 
the approach, methods, workplan and management arrangements proposed for the 
evaluation and describes possible team members and the proposal for selection of 
country research teams: 


The ‘Co-Financing Programme (CFP) Evaluation 2003-2006’ sets out an ambitious long- 
term research agenda, that involves the Government CFP and related bodies, the CFAs 
and their partners at country level. The background paper developed bythe Carer nary 
aims to promote a ‘learning organisation’ approach and M&E capacity ee 


both centrally and at country level. 


The health programme evaluation’s overall objective is to gain insight into the 
contribution of the CFAs and their partner organisations to the improvement ns 
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le use of, access to and control over health-related services for the poorer = 
sections of the population, with special emphasis on women and youth: As describe in 
the background paper, the CFA evaluation is concerned mainly with programme inputs, 
processes, outputs, and outcomes. Impact, in terms of demonstrable and attributable 
changes in people’s health and poverty status, is understood to be difficult to identify in 


this study, but will be assessed where possible. 


sustainab 


There are a number of challenging issues for consideration during the evaluation that 
are raised in the background paper and in the November 2002 evaluation report of the 
Co-financing Programme. The most significant are discussed below. 


Levels of analysis 
The background paper describes four core ‘units’ or levels of analysis: members of target 


groups; partner organisations; civil society; and the CFAs. The framework will address 
all four levels, ensuring the proper disaggregation of factors influencing process and 
results, and identifying any cause/effect relationships. 


It will also be necessary to include the perspectives of the Netherlands development 
agency at country level as well as globally, in terms of the official country strategy and 
wider health sector support, and those of the officials managing the CFP itself. A key 
step in the inception stage is to develop a full understanding of current and emerging 
aid policy in the Netherlands. The coherence of the Co-financing Programme and CFA 
strategies with national and international policy objectives is an important issue to 
explore. Likewise, national government perspectives and fit with national health and 
HIV/AIDS strategies will be important. 


It will be important to map key players, and to carry out stakeholder analysis, in order 
to generate an overall assessment of the context in which the partnerships are taking 
place, as well as detailed analysis of the selected programmes, and their relationships 
to wider policy and operational processes (e.g. district priority setting and planning, 
resource allocation and health financing policies). 


Process versus results and types of engagement/intervention 

The 2002 evaluation suggests that, of the three priority interventions, the focus 

of programme efforts in the past has tended to be more on service delivery and 
organisational capacity building, as opposed to civil society development (citizen voice, 
policy advocacy etc). However, the background paper implies that the evaluations 
themselves have perhaps failed to capture the complexity of the relationship between 
service delivery and organisational development, with processes for policy-advocacy 
and civil society development. This relationship is also changing rapidly due to the 
factors outlined in the previous section. 


In the report and background paper, there is some emphasis on qualitative and 
quantitative indicators of ‘reach’ (numbers and type of people in poverty). Under- 
standing the differentiation of target groups, and actual versus targeted reach is clearly 
important. However, it can be over-emphasised, at the expense of assessing capacity and 
impact of influence on policy and its implementation. 


These issues are recognised as significant challenges for organisations that are shifting 
their focus from pro-poor service delivery for a comparatively small number of people, 
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to efforts to develop credibility and influence of 


; Civil society in the wider polic 
environment, with a view to increasing impact iid 


for poor people in the longer term. 


The framework will therefore include refere 
for monitoring and evaluating the policy-a 
recent international NGO work on process 
effectiveness of policy influence. Effective 


and a strong evidence base with clients and communities, and working with them to 
develop their case and use their voice effectively for policy change - with local and 
national representatives and officials, other CSOs and the media. This also critically 
requires action on policy implementation and monitoring, aspects which are sometimes 
neglected. As pointed out in the background papers, intermediaries have a critical 

role in this process, but the extent to which they are perceived as having legitimacy 


with communities and credibility among policy makers and other advocacy targets are 
important issues for assessment. 


nce to recently developed methodologies 
dvocacy-evidence loop. It will draw on 
indicators with which to measure the 
process relies on developing credibility 


Results-based framework for performance assessment of CFAs and partners 


The 2002 evaluation recommended that CFAs develop stronger approaches to 
monitoring and evaluating the performance of both themselves and their partners. It 
suggests a result-based, ‘owned’ framework that set outs vision, strategic objectives and 
activities that have been developed through consultation between CFA and partners, and 
moves away from a focus on financial accountability. 


The degree to which this has been possible or acceptable will undoubtedly have varied 
in the three CFAs and their partners. An important step during the inception phase will 
be to review the extent to which this has happened, and the process by which it took 
place. It will be important to use (and develop if necessary) such a framework with the 
partners (and the CFAs) in order for them to use for recording their own progress against 
their vision and strategy, both for the evaluation, and indeed for the future. 


Evaluation strategy and methodology 


General comment on approach 

IHSD proposes using a participatory approach to this evaluation, whereby the three CFAs 
and partner organisations are involved in the design of the TORS, and in developing 

the overall framework for assessment, as well as in reviewing the initial outputs of 

the evaluation at country level. An inception meeting is proposed for the CFAs in The 
Netherlands, together with pre-implementation meetings in the three countries. A final 
meeting in each country is proposed, to review results of the evaluation, and consider 
how to ensure that conclusions and recommendations are fed back into the planning 
process. Full engagement of the CFAs and partner organisations will be maintained 
through the evaluation management and communication mechanisms (see following 


sections). 


‘A general overall framework will be developed in conjunction with headquarter staff of 
the three CFAs during the initial inception period, while staff and partners within each 
country programme will be able to refine various aspects of the evaluation according to 


their specific country situation and needs. 
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CFAs, this evaluation will take the opportunity 


A partner organisations in the selected 
tative in the 


Given the capacity building focus of the 
to enhance evaluation skills amongst CH 
countries. This will be achieved through involving a partner represen 


evaluation consultative group, to be set up by IHSD. 


Conceptual framework . | 
Initial thoughts on the conceptual framework underlying this evaluation are provided 


here. These will need refining based on the first phase of the evaluation process itself, 
which would involve reviewing policy and project documentation, previous evaluations 


and interviewing headquarter staff. 


The results-based variables that this evaluation will look at are (at outcome level): 
change/improvements in use of, access to and control over health (-related) services, by 
poor people (the reach, involving identification of the beneficiaries, as far as possible). 
Analysis of impact, in terms of attributed change in health or poverty status, is probably 
beyond the scope, but data will be collected where possible. 


Programme outcomes are delivered or mediated through a number of processes or 
outputs, including direct service provision to poor communities, civil society building 
through development of alliances and networks, citizen and client voice, and policy- 
advocacy activities, often undertaken by the intermediary partner organisations. 
Investments/inputs to these processes by the CFAs are the key strategies of a) working 
in partnership with a number of groups in each country, and b) facilitation of 
organisational development or capacity building. 


When looked at as a system, these various aspects come together as in the diagram. 


Processes and results: CFA inputs, partnership processes, and programme outputs and outcomes 


Inputs Processes of outputs Outcomes 

Partnership processes Services provision Use of; access to; control 
- agreements and over; health (related) 
ways of working Civil society building resources and services 


AND Organisation by the poor (reach to be 
development Policy-Advocacy differentiated) 
processes 


Evaluation 


The framework developed will need to address issues such as: 


° Effectiveness of projects - whether the projects have achieved their objectives, 
including evidence of impact on health of target groups and of impact on the 
position of men and women; 

Sustainability and influence - whether completed projects have proved to 

be sustainable, developed capacity and/or influenced policy processes or 
implementation; 

Coherence - whether the combination of health support to the country is coherent 
(across bilateral and NGO CFA activities) and fits within The Netherlands priorities 
for engagement in that country; 
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such as in Nepal or the DRC. 
e Process - how well the design and implementation 
participation and ownership; 
¢ Quality of design - both technical as 
the support has addressed HIV/AIDS, 
e Consistency with Netherlands health 
and emphasis on public health; 


e Efficiency - whether the support has offered good value for money. 


of support built in stakeholder 


pects of the health interventions and how well 
equity, poverty and gender issues; 
Priorities - how the support fits with policies 


Methodology 

The evaluation methodology will combine quantitative and qualitative methods to explore 
the variables (indicators) identified as key to this evaluation. To a large degree these 
indicators will be process and output orientated, due to the nature of the questions being 
asked. Where reliable time trend data exists, evaluators will also examine outcome and 
impact indicators as well. The table outlines how we would propose to answer the main 
questions posed in the terms of reference for the evaluation. 


Main questions to be addressed Methodology _| Evaluation Activity 


A - Which visions, strategies and activities | Qualitative ¢ Document review 
¢ Interviews with staff of CFA and partner organisations 


do partner organisations have to improve 
access to, use of and control over basic 
health services by potential users; 
a Quantitative Time trend analysis of health status data in specific project area 
(if available) compared to other parts of the country; 
¢ Process data from project areas - who attends clinics, who 
serves on management committees etc.? 


B —- What are the qualitative and 
quantitative results of these programmes 
and for whom do they improve sustainable 
access to, use of and control over basic 
health (-related) services; 


e Focus group interviews (gender and age specific groups) in 
project areas 
e Interviews with health staff, health management committees, 


local government staff; 


b Qualitative 


they are doing against agreed criteria, and areas for further 
improvement. 


e Interviews with MOH and other national bodies to receive 
qualitative comparison 

C and D - To which extent do complement- | Qualitative e 

ary partnerships, participation in networks e 

and/or alliances contribute to better per- e 

formance of partners and health systems? 

AND With what results do (complementary) 

partnerships, participation in networks 

and/or alliances contribute to a better 

performance of the partner organisations , 

and the health system in general? 
Input review - coordination of inputs | . 
Process review - contribution of ways of working to improving 
internal and external processes within individual partners, plus 
assessment of partnerships themselves (using established tools) 
Interviews with partners on how the support has contributed, 


Document review (project reports, minutes from meetings etc.) 
E - What is the added value, in terms of Qualitative ° 
e 
and what would have done in its absence 
outputs 


self-assessment tool, partner organisations assess how well 
their programmes? 


Interviews with local and national level stakeholders 
Partnership assessment workshop - using well established 
results and partnership aspects, of the CFAs 
support for the partner-organisations and 
¢ Output review - contribution of ways of working to project 
© Outcome review - where data exists to provide attributable 
results. 
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Workplan and schedule for the evaluation 


Team make-up, roles and responsibilities | . 
We propose that the evaluation be carried out by an international and national team, 


made up of national researchers in each of the three countries. The international team 
leader will coordinate developing the overall framework for the evaluation, maintain 
oversight of activities and quality assurance as well as lead on report writing and 
synthesis. The second international consultant will contribute to developing the 
framework and methodology, carry out initial interviews in the Netherlands, and prepare 


the preliminary desk study. 


At country level, a lead national researcher will lead a team of two or three people to 
undertake country studies, interviews and focus groups, possibly under the aegis of a 
research institution or expert consulting group. They will also take part in reviewing 
and adapting the methodology, the national overview studies, in conducting stakeholder 
interviews, and be involved in focus group discussions, final partner workshops and 
write-up of a country report. 


At the beginning of the study, a workshop/meeting will take place with the team, the 
partners and the CFA representatives. The partners will also be supported in preparing 
their own assessment of progress against an agreed template framework. A second 
workshop will take place during the Consolidation Phase to review preliminary findings 
and conclusions, and any emerging implications for future strategy. 


To support the national teams, two visits by the international consultants will take 
place in each country. The two-person international team will both take part in the first 
workshop in one country to ensure an agreed understanding of approach and activities, 
and overall methodology, for adapting with partners in the remaining two countries. 
The team leader will continue providing support to that country, visiting once more 

for the final workshop. Each international consultant will work with teams in the two 
remaining countries, and visit the second workshop. It is also planned that at least one 
national researcher shall attend the final meeting in the Netherlands. 


The detailed workplan is outlined below. 
Inception phase (July/August) 


° Review of project documentation, development of the background papers/desk study, 
review and development of draft framework for the evaluation 

* Final shortlisting, selection and briefing of country teams 

¢ Interviews with key staff based in the Netherlands (CFA health reference group and 

agencies, CFP steering committee and relevant Government bodies) 

Country mapping of CFA programmes, development of selection criteria and (in 

liaison with CFAs) inviting partners to participate 

An inception meeting with all three CFAs to agree on refined terms of reference and 

overall framework for the evaluation and the evaluation plan 

aaa methodology and tools will also be developed for review by CFAs and national 

eams 

The evaluation consultation group will be established, to include evaluation team and 


ea representatives (see management section 6). The evaluation website will also 
e set up. 
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Implementation phase (August/September) 
This phase will be made up of a number of different activities. 


The first activity will be holding a count 
have input into the evaluation process b 
receive guidance/pointers about who sh 
investigation. 

Partners will also be invited to develo 
evaluation data collected during the 
performance framework. This will be 
At the same time, the collection and 
to be compared with project data. 
Focus group discussions and stakeholder interviews will follow, using a semi- 
structured guide to include questions common across all three evaluations plus 
specific questions developed by partners in that country. 


ry level pre-implementation meeting to 
y Participating Organisations, as well as to 
ould be interviewed and unique factors for 


p their own analysis of monitoring and 

project life, set out according to the agreed 

for review and discussion with the team. 
analysis of in-country health data can take place 


Consolidation phase (October) 


Initial results of data collected so far will be synthesised and presented at a partner's 
workshop in each country. 

This workshop will also be used to give time to members of the partnership/ 
network to assess how they feel the partnership is working and areas that could use 
improvement or strengthening, and implications for future strategy. 

An interim report for each country will also be provided by the team leader to the 
evaluation consultative group. 


Finalisation (November/December) 


The lead national researcher will finalise the country report, in consultation with all 
members of the evaluation team, and under guidance of the evaluation team leader 
The evaluation team leader will draw together all three reports, and the international 
desk study and interviews, into a summary evaluation that synthesises lessons 
learned across all three countries, and emerging issues for policy and strategy. This 
will be developed in a format for use at country level, by partners, as well as by the 
CFAs and for the CFP. 

A final meeting with the staff of the three CFAs will be held to provide an overview 
and an opportunity for further comments and clarification before the report is 
finalised. At least one research lead will also attend the workshop 

The report will be submitted to the CFA Health Evaluation Reference Group, for final 
comments (for incorporation by the team by end January 2005). 
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Appendix 4 
Background Document 
MBN Coordination Group Evaluation Health Theme 


The health sector and related working areas are important areas of intervention for 

the Dutch Co-financing Agencies, both in number of partner organisations and in the 
amount of funding concerned. Therefore three Co-Financing Agencies (CFAs), Cordaid 
ICCO and Plan Netherlands have decided to commission a programme evaluation to : 
gain better insight in the results of their support of health (related) activities for poverty 
reduction and to learn from their practice. The evaluation is seen in relation tot the 
ongoing evaluation efforts related to the CFP (Co-Funding Programme) since 1980. In 
the annex to this document a list of relevant documents is included. 


The earlier evaluations showed the following. 

e In general the programmes effectively contributed to improved access to health 
related services for poor groups, especially in areas in which the government 
provides no, little, or too little health care. The quality of the care provided is good 
and appreciated by the population. The co-operation and division of tasks with local 
authorities, village leaders and other NGOs is, in general, good. 

e Other results are a better-informed society and higher levels of involvement of 
local communities and schools in health and health related services, innovative 
care related to Hiv/Aids such as counselling, prevention and training of traditional 
healers. 

e The contribution to overall civil society building and lobby & advocacy, however, is 
not something that is pursued and was found to be limited. 


There are some remarks to be made regarding these conclusions. The evaluations failed 
to observe and analyse support and network organisations in the health (related) sector 
that were supported by the CFAs and to which the partner organisations involved in 
the research were affiliated. In the countries concerned, a division of tasks between 
organisations regarding service delivery and influencing policy is / was regarded 

as normal, especially by the church-related organisations. By zooming in on the 
grassroots level, it was no surprise that issues related to lobby were underexposed while 
direct services to the population came out clearly. Furthermore, the health (related) 
interventions that were studied under the CFA evaluation programme 1999-2002, were 
conceived in the mid 1990s. Emphasis regarding approaches, strategies and activities 
have shifted since then, as shown in the next section. 


The implication of the above is that the present evaluation must give attention to 
both direct service delivery and to civil society building aspects of the CFA supported 
interventions in the health sector. 


CFA support to health related activities 


The CFAs view the access to services and the promotion of the right to healthcare as a 
basic condition for socio-economic development. By doing so, they expect to contribute 
to the realisation of several Millennium Development Goals by funding health (related) 
programmes. Health (related) services are interpreted in a broad sense, including 
community based health care, hygiene education, water and sanitation et cetera. Many 
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health related activities are part of broader, multi sectoral programmes which often 


include food security, income generation, water and sanitation aspects. ee 
By supporting initiatives in the health (related) sector, the three CFAs emp asise tha 
besides the need to improve quality of and access to health (related) services, also 
participation of the target group in control over the services 1s important. To promote 
such control it is necessary to increase the claim making capacity, participation in 
(quality) control by the users of the services and lobby. These aspects of civil society 
building (CSB) are part of the so-called rights based approach, which gives normative, 
strategic and operational direction to the programmes. CSB is considered strategic for 
the realisation of access to and control over services in the health sector. 


The intervention strategies applied by Cordaid, ICCO and Plan Netherlands are the well- 
known threesome of ‘direct poverty alleviation’, ‘civil society building’ and ‘influencing 
policy’ related to the Dutch CFP. Many health (related) programmes supported by the 
CFAs, specifically target women and girls. As in all programmes and sectors the gender 
aspect is of importance, and especially in areas as reproductive health care the gender 
dimension is relevant. 


In recent years the health sector has been subject to two major changes. One is related 
to the decreasing role of the state in relation to health and the second is related to the 
increasing role of the private sector. The combined effect of this development obviously 
has an impact on the degree of access to and the quality of health (related) services for 
the poorer sections of the population in the countries concerned. Therefore increasing 
attention is given to the response capacity of partner organisations to these changes, 
and to collaboration with the public and the private sector. The defining of the role of 
civil society in the health sector (complementarity towards the state and private sector) 
is different per region. This also affects the choice partner organisations make in terms 
of their programmes: they may concentrate on direct service delivery, civil society 
building aspects, or a combination of both. 


The characterisation of the health (related) programmes given below is based on two 
dimensions: a) policies, priorities and approaches, and b) funding. Of each organisation 
several policy documents are available and can be used for the actual study. All three 
organisations support broad integrated development programmes, often in a rural 
context with a health care (related) component. On the other hand the CFAs also fund 
sectoral health (related) programmes. 


Cordaid 


a Policies, priorities and approaches 
Within the health and care theme Cordaid aims at improving the health situation 
and the quality of and access to health and care for poor and vulnerable groups. 
Cordaid sees the improvement in health not only in increasing the access to health 
services for poor people, but also strengthening participation by the population in 
the management of and control over health care resources and programmes. An 
important focus is the strengthening of local organisations and encouraging co- 
operation between various actors. The general strategy for health and care is to 
work on the three levels of community, service delivery and national networking. 
These three levels are linked with the general Cordaid strategies of direct poverty 
alleviation, civil society building and lobby. 
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The most typical partner Organisations j 
provision NGOs, including church or 
for these organisations is directed to 


resource development at different levels is an 
od Cordaid has provided personnel support, but 
hnical assistance for long-term self-reliance for 


important strategy. For a long peri 
is now shifting its attention to tec 
human resources. 


b Funding 


Cordaids funding, expressed in approved contributions per year, varied during the 
last three years from € 36,3 million (2000), to € 30,7 million (2001) and € 32,5 
million (2002). Most financial resources for health (related) programmes are spent 
to Africa (49% of commitments approved in 2002), the second most important 


destination is Asia (31%), followed by Latin America 11% and Central and Eastern 
Europe 9%. 


ICCO 


a Policies, priorities and approaches 
Health (related) programmes are part of one of ICCO’s three main policy themes 
‘access to basic social services. ICCO’s policy paper (April 2003) carries the subtitle 
‘adopting a rights-based approach in basic services’ which confirms an important 
policy shift in the last few years. Whereas the actual supply of basic social services 
made up a considerable share of the ICCO supported health programmes in the past, 
the present aim is a combination of service delivery and the empowerment and 
mobilisation of people. Both are strategies in the struggle to acquire rights for the 
increase of access to services of excluded or disadvantaged groups. 
ICCO supports activities such as health education, the training of village health care 
workers, traditional birth attendants, and simple curative care, often as elements 
of broader integrated rural development programmes. The largest share of the 
beneficiaries of the interventions consists of women and girls. Increasingly issues 
related to Hiv/Aids, reproductive health & rights, institutional strengthening and 
capacity building as well as lobby & advocacy receive more funding. The Christian 
Health Associations in Africa occupy a specific position. Their contribution becomes 
more important, also to the international donor community, in the present context 
of decentralisation, sector wide approaches and PRSPs. ICCO supports these é' 
organisations to respond adequately to the resulting changes in their institutional 
environment. Another area of special interest is the position of churches regarding 
Aids. For the support of health (related) activities, ICCO disposes of a department 
for technical co-operation as well as an advisory service to complement the funding 
resources. 


b Funding 
ICCO’s funding, expressed in amount transferred per year, has increased pega hii 
last three years from € 5 million (2000), to € 7,7 million (2001) and € 9,8 — 
(2002) a year. Most financial resources for health (related) gh yer 9 go to Africa 
(44% of the transferred amount in 2002), the second most important destination Is 
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Asia and Europe (31%), while Latin America receives 16% of the funds meant for 


health (related) activities. 


Plan Netherlands 


a Policies, priorities and approaches | 
The ‘growing up healthy’ domain of Plan International covers healthcare and 


activities related to Hiv/Aids and water supply and sanitation. The focus is on mother 
and child care and special attention is given to reproductive health, transmissible 
diseases and nutrition, and malaria. Plan Netherlands supports mainly activities at 
the community level, involving community-based organisations, local NGO’s, local 
and national government. Plan works from a rights perspective. This implies not 
only contributing to improving the quantity and quality of health services, but also 
increasing local people’s participation in healthcare, strengthening the ability of 
community-based and other civil society organisations to claim health rights and 
also enhancing the accountability and responsiveness of health service organisations. 
In addition, program support is expected to directly or indirectly contribute to the 
elaboration and implementation of pro-poor health policies and strategies at the 
national level. 


b Funding 
Allocation of funds to health-related projects and programmes varied from 24% 
in 2001, 27% in 2002, and 21% in 2003 to 19% in 2004. After education is health 
the largest ‘sector’. For 2004 a total amount of 3.839 million Euro was budgeted for 
health-related programs in Africa, Asia and Latin America. 


Objective of the present evaluation 


The general objective is to gain insight in the contribution of the co-operation of CFAs 

and their partner organisations to the improvement of sustainable use of, access to and 
control over health (related') services? for the poorer sections of the populations with a 

special emphasis on women and youth. The analysis includes process? (how) and results 
(what). 


The study of ‘access to’, ‘use of and ‘control over’ requires emphasis on the notion of 
direct and indirect ‘reach’, including 
* Qualitative aspects, such as differentiation of population reached, quality and nature 
of the services, links between the delivered services and the existing health sector. 
Quality should be assessed in the broader context of the national health systems and 
by relating the CFA contribution to that of other actors involved in health support.‘ 
* Quantitative aspects, such as number of users, cost effectiveness. 
Financial, social and institutional sustainability aspects, such as user contributions 
and insurance systems, involvement of government institutions and other possible 


‘ An example of a related service is the provision of clean drinking water, that helps to improve the general 
health condition of the population concerned. 

’ Services must be understood both in qualitative and in quantitative terms. 

* Giving attention to roles of stakeholders involved, complementarity of these roles and dilemma’s in 
choices to be made concerning those roles. | 

* The analysis should be done from the perspective of the partner organisations and / or the programme, for 
example by a concise overview of literature concerning the health sector in the country concerned. 
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Four levels of analysis are involved 


market actors in the health sector, res 


et cetera. Ponses to Hiv/Aids problems, donor dependency 


(covering three links in the aid 
chai 
(Members of) target group(s): see questions above . 


Partner organisations: including relations with other relevant ‘actors’ in the context 


concerned, policy and strategy issues, learning capacities, collaboration with CFAs 
Civil society, including networks, platforms, ete. 
CFAs: collaboration with partner or 


ganisations, policy and strategy issues 
roles (donor +). gy , multiple 


Main questions 


Which visions, strategies and activities do partner organisations have to improve the 
access to, use of and control over basic health (related) services by (potential) users in 
the programmes funded entirely or partially by the CFAs? 

What are the qualitative and quantitative results of these programmes and for whom 
(differentiation of the benefited groups in society) do they improve sustainable access 
to, use of and control over basic health (related) services? 

To which extent do (complementary) partnerships, participation in networks and / 

or alliances contribute to a better performance of the partner organisations and the 
health system in general? 

With what results do (complementary) partnerships, participation in networks and / 
or alliances contribute to a better performance of the partner organisations and the 
health system in general? 

What is the added value, in term of results and partnership aspects, of the CFAs 
support for the partner organisations and their programmes? 


In these questions both the intended and de unintended, the explicit and implicit effects 
must be taken into account. 


Choice of the research area, i.e. countries 


The three CFAs have formulated the following criteria for the selection of the research 
areas: 


Substantial financial CFA contribution to health related programmes 

Considerable number of partner organisations active in the health (related) sector , 
Existence of a network, support organisation or other formal or informal structure in 
the health system in which the partner organisations participate in order to undertake 
lobby, advocacy or Other activities or receive support from it 

Evenly spread over the continents* 

Not too many evaluations in the past few years 


5 It was not possible to identify a country in Latin America which fitted the other criteria 
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with health (related) programmes of the 


it i find overlap in countries ’ nigees 
Because itdsparcite f tudy and compare their activities. Based 


participating CFAs, a 3x2 model is chosen to s 
on these criteria the following countries are selected. 


Cordaid 


Country 


The chosen model uses the available means (time and funds) for this study efficiently, 
because it permits the comparison of two CFA supported health (related) programmes in 
each of the three countries. The selection of the programmes will take place in dialogue 
with the research co-ordinator. Criteria correspond to: 

e Select a mixture of different type of programmes (integrated, sectoral, lobby & 
advocacy), including only organisations that have (an) objective(s) related to health 
care® 

e Balance rural - urban (if any) 

e Include programmes operating from 1999 onwards (minimise overlap with previous 
evaluations) 

e Exclude partner organisations were informed in writing and before December 2003 
that the relation with ICCO, Cordaid or Plan Netherlands would be ended’ 


Indicative planning (see also letter) 


e The research starts as early as possible (June-July) in 2004. The first phase consists 
of the further elaboration of the research plan, formulation of the Master Terms of 
Reference and the carrying out of the desk study in the offices of the three CFAs. The 
desk study produces insights into the relevant information and serves as input for the 
Terms of Reference for the three country studies. 

e By July 2004 the CFAs will approve the four ToRs (one master ToR and three ToRs for 
the country studies) and the choice of local researchers has been made. 

e In August- October 2004 the fieldwork in the different countries takes place. 

e The draft version of the report will be ready in December 2004. 

¢ The final report will be assessed, and if found satisfactory, approved by the CFAs end 
of January 2005. 


° This is to exclude programmes or organisations that work completely outside the health system, but 
that indirectly contribute to the health situation by improving other aspects of livelihoods (for example: 
programme aimed at improving agricultural production and by that contributing to the health situation). 
’ This is to exclude partner organisations that do not fit (any more) in the CFA programmes, to avoid 
conclusions already out dated by decisions taken by the CFA and/ or partner organisation. 
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Evaluation Analytical Framework 


Characteristics (Social, 
economic, structure, 

Status, attitudes, type of 
organisation) 


Differences Implicationsfor 
individual 
programmes, and 
CFP 
Context (societal, political, 
CT A SS clue 


Policy themes/domains 

e Access to basic social 
services 

¢ Fair economic development 

¢ Democratisation & Peace 
building 

© Civil Society Development 

¢ Care for and rights of 
vulnerable groups 

© Community-based health 

care 

Reproductive Health 

HIV/AIDS 

Environment 

Basic education 

Advocacy (type and level) 

Food security 

Habitat 


Sensitive to and respect for 

cross-cutting issues as 

© Poverty alleviation 

e Empowerment and 
sustainability 

© Gender equity 

¢ Environmental 
sustainability 

© Good governance 

© Behaviour change 
communication 


Target groups eh ee 
Projects & partners 


Alliances/networks 


LenijepimiésDiuilide Tonsil (1002. 
plan FOE Naticn) | 
Organisation 
¢ Monagement structure 
© Capacity building 
* Communication systems 
e MéeE 


Cordaid (Memisa, 
Mensen in Nood, 
Vastenactie) 


Common 
Features 
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CFA partner information sheets 


Name of partner organisation 


Type of organisation (national NGO, CBO, 
Network etc.) 

Aims, objectives and strategies of 
organisation 

Aims, objectives and strategies of specific 
projects (if applicable) 

Aim of CFA support to the partner 
organisation (programme, capacity 
building etc) 

Reason partner organisation was chosen 
to be a partner of the CFA 

When did support start? 


Where geographically do CFA funded 
activities to the partner operate eg a 
specific rural area (if different)? 


Proportion of CFA support given to the 
CFP priority ‘three interventions’ of 
partner organisation (ie services versus 
institutional development versus civil 
society development) 


Types of activities in the health or 
related sector 


Percentage of activities done in the health 
or related sector 


Activities linked to health aspects of 
HIV/AIDS and/or undertaken by the health 
sector (and any links with mainstreaming 
activities) 
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List of CFA Policy Documents Consulted 


General 

e Background paper for evaluation 

e MFP Breed Beleidskader en Subsidieregeling 

e Policy Framework for the Broad-based Co-financing Programme (MFP-Breed) 


Cordaid 

2003 Annual Report, 2004 

Health and Care policy paper 2003-2006 

Cordaid Health Policy Malawi-Zambia 2003-2006 
Partner meeting in DR Congo, 2003 

Health Care & technical assistance: a pertinent question 
Evaluation BDOM Kananga, Bukavu, Kinshasa 
Projectkenschetsen D.R. Congo en Zambia 

D.R.Congo, Rwanda en Burundi Beleidsplan 2003-2006 


ICCO 

Policy Choices and Implementation 2003-2006 

2003-2006 Business Plan 

Stramien Landenjaarplan Congo 2002, 2003 

Azeo Jaarplan 2004-11-16 

Landenplan ICCO Nepal 2002 

Landen Jaarplan D.R.Congo2004 

2001 Annual report 

Policy papers Fair economic Development, Gender Policy, Democratisation & peace 
building, Human Rights, Access to Basic Services, Partnership Policy 
Reisverslag Ger Kuiper DR Congo 

Kenschets aanvragen Nepal, D.R.Congo 

ICCO Partner Consultation 2002 

OD Approaches, 2002 

Covernota Grote Meren Gebied 2001-2003 


Plan 

Business plan 2003-2006 

Annual report 2003 air. 
Policy papers: Civil Society Development, Gender Equity, Partnerships, Health, Child Rights 
Country Programme Outline Zambia (2002-2005) 

Regional plans 2003-2006 (Africa) 
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Appendix 7 


Interview Guide 


List of people/organisations to be interviewed 


National level 


Plan International - Country Director/Programme Manager 

Church Health Association - Health Programme Director 

NGO coordinating body - Director or Coordinator 

Other National level civil society groups 

Ministry of Health - relevant individual indicated by partners 

Ministry of Local Government - where local government also handles health 
National AIDS Committee - Director or Programme Manager 


District level 


District Health/Medical Officer 

District NGO coordinating group 

Diocesan leader 

Health centre/Hospital lead staff (director, medical assistant, head nurse) 


Partner organisations 


Director 
Health Programme Manager (if exists) 
Education and Training lead (if exists) 


Partnership members 
Directors or programme managers of a selected group of partners 
Beneficiaries 


For each project where the focus is on health service delivery - Focus group discussions 
(no more than 8 people to a group) with a minimum of: 

1 woman’s group 

1 men’s group 

1 youth group 

1 elderly group 
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Questions for key national and district stakeholders 


(Ministry of Health, Ministry of Local Government, National Church Associations, 


National Civil Society/NGO Coordination Group) 


1 What do you know about the health and/or advocacy programme of Plan and (insert 
partners - dioceses, Church Health Associations, local NGOs etc.)? 


2 What is the relationship of this programme to the national policies of your 
organisation/ministry? (programmes are within the national/district plan? Meetings 
with ministry, district or umbrella organisation staff? Etc.) 


3 What impact has this programme had on access to, use of and/or control over health 
services by potential beneficiaries? 


4 Has the work of this programme had any influence on developments of national, 
district or local policy or on civil society programmes that you are aware of? 


If yes, what has this influence been? 


If no, do you think the programme should be more influential than it is? What 
should or could Plan or partners do to raise the profile of their work? 


5 What should or could Plan or partners do to raise the profile of their work? 


6 Any other comments you would like to make? 
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Questions for partner Organisation staff 


Where possible, first have partner staff fill in 
to get their views of how well the organisatio 


1 


10 


11 


12 


the organisational analysis questionnaire 
n ls working. 


General background questions: What is 


your role in this organisation? How lon 
have you worked for this organisation? : ’ 


What are the health activities supported by your organisation? How are they funded 


(donor and percentage of overall funding). What specific components does (Cordaid/ 
Plan/ICCO) support? Has this changed over time? Why? 


What is the vision and objectives of your organisation related to health and/or 
advocacy work? 


What strategies and activities does your organisation undertake to ensure there is 
gender equity in access to, use of and control over health services? 


What strategies and activities does your organisation undertake to ensure 
participation of beneficiaries and the wider community in your health interventions? 
Do beneficiaries and community members participate in decision making about your 
interventions? 


When considering your health service or advocacy-related programmes, what do 
you think is working particularly well for improving people’s access to, use of and 
control over basic health services? What are your main achievements in this area? 
Why? (try to get concrete examples or stories that you can use in your report) 


What do you think is working less well in this area or needs improvement? Why? (as 
above, try to get concrete examples) 


Are you a member of a partnership or network? If yes, what contribution has 
your organisation made to that partnership or network? How have you made that 
contribution? 


Has working with partners helped your organisation to improve access to, use of and 
control over health services by beneficiaries? If yes, give examples of how this has 
happened. 


What are the constraints or challenges of working in partnership? Why? 


Besides the financial contribution that your Dutch partner makes to your | 
organisation, what have been the other benefits and challenges of working with your 


Dutch partner? 


Any other comments you would like to make related to the above? 
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Questions for members of partnership 


the partnership assessment form first, 


ossible, please have the interviewee fill in 
as : iven in the form to help prompt for 


before asking questions. You can use the answers g 
more detail in the interview. 


1 What is the vision, aim and objectives of the partnership in relation to improving 


access to, use of and control over health services by beneficiaries? 


2 How did the members of the partnership come to agree with its vision, strategies and 


activities? 


3 Has the partnership achieved successful outcomes yet? 


If yes, what have these been? What do you think contributed to the success? 
If no, what have been the constraints to successful achievement? 


4 To what degree has working in partnership with other local organisations helped to 
strengthen your own organisation? (Please try to get concrete examples) 


5 What contribution does your organisation make to help strengthen the partnership, 
and other members of the partnership? Please try to get concrete examples) 


6 What is the role of ICCO, Cordaid or Plan in your partnership? 

7 How do you appreciate the contribution of ICCO, Cordaid or Plan to the work of your 
partnership? 
What is working well? 


What could be improved? How could it be improved? 


8 Any other comments you would like to make? 
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Guideline for focus group discussions 


No more than 8 to 10 participants in each group. At a minimum try to create groups of: 


1 woman’s user group 
1 youth user group 


1 men's community group (not necessarily service users) 
1 woman’s community group (not necessarily service users) 
1 youth community group (not necessarily service users) 


Welcome participants, ensure everyone understands w 
are happy to participate in the discussion. Ask everyone to feel free to contribute, and 
ensure they understand that what they say will not be attributed to them in any report. 
The purpose is to get an understanding of what people like them think. 


hy they are there and that they 


1 What are the most important health problems in the area? Why? 


2 Who do most people go to when they or a family member is ill? Why? 


3 Assess impact of the project or health service as far as community members are 
concerned using any number of participatory methods, e.g. Create an impact 
flowchart of what impact they feel the project or health service is having in their 
area. An impact flowchart starts with an event or action, such as setting up a new 
health clinic, and then explores the consequences by asking ‘what happened next?” 
Or use a spider diagram to visualise how access and use of health services have 
changed from before the project or health service began to now. 


4 Who is involved in managing the health service/project? How are they involved? 


5 What areas of the health service or project are working well, in your opinion? 


6 What is not working so well? Why? How could it be improve? 


2 Roche C (1999) Impact Assessment for Development Agencies. Oxford: Oxfam Publications 
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Appendix 8 
Organisational and Partnership Assessment Tool 


(Adapted from Centre for Advancement of Co 


llaborative St 
New York Academy of Medicine) © otrategies for Health, 


This questionnaire asks questions about different aspects of your organisation and 
partnership arrangements. The questionnaire allows you to express your opinions and 
provide information about your experiences anonymously - your name will not be used 
in any way in connection with your responses. There are no right or wrong answers 


to the questions. Thoughtful and honest responses will give your partnership the most 
valuable information. Please answer every question if you can. 


Leadership: Please think about all 
the people who provide formal 
and informal leadership in your 
organisation. Please rate how well 
you think these people 


Empower people involved in the 
Organisation 


Communicate the vision of the 
organisation 

Work to develop common language in 
the organisation 


Foster respect, trust, inclusiveness and 
openness 


Create an environment where 
differences of opinion can be voiced 


Resolve conflict amongst staff 
members and others 
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Very good 


EVALUATION OF HEALTH 


Please choose the 


Efficiency: 
cribes what 


statement that best des 
you think about: 

How well your organisation uses its 
financial resources 

How well your organisation uses its 
members and/or the community's skills, 
information and connections. 


How well your organisation uses its 
staffs and members time 


Administration and management: 
now we would like you to think about 
the management and administrative 
activities in your partnership. Please 
rate the effectiveness of your 
organisation in carrying out the 


following: pape 
Coordinating communication within 
the organisation 


Coordinating communication with 
people outside the organisation, and 
with other organisations 


Preparing materials that inform staff 
and members and help them make 
decisions 


Evaluating the progress and impact of 
the organisation's work 


Minimising barriers to participation 
in the organisation's meetings and 
activities (e.g. providing child care 
for mothers, or transport for people 
travelling from a distance) 
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Non-financial resources: an 
organisation also needs a number 
of non-financial resources to work 
effectively and achieve its goals. 
For each of the following types of 
resources, to what extent does your 
organisation have what it needs to 
work effectively? 


All of what it 


Most of what | Some of 
alias of what | None of what 


it needs it needs it needs 


Skills and expertise (e.g. leadership, 
administration, evaluation, clinical, 
training, community organising) 


Data and information (e.g. statistical 
data, information about community 
perceptions, values, resources and 

politics) 


Connections to the beneficiary 
community? 


Connections to political decision 
makers, government agencies, other 
organisation/groups 


Financial and other capital resources: for 
each of the following types of resources 
to what extent does your organisation 

have what it needs to work effectively? 


Equipment and goods 
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Partnerships and networks 


urrently involved in related to 
None of 
the time 


Now think about partnerships and networks that you are c 
health service delivery, please answer the following questions: 


All of the Most of Some of 
time the time the time 
Are you comfortable with how decisions are made in ae 
the partnership. . . 
How often do you support the decisions made by the me 


Decision making 


Almost none 
of the time 


Extremely | Very well | Somewhat 
well well 


ys y 


Not well 
at all 


Synergy: thinking about the people and 
organisations that are participants in your 
partnership 


By working together, how well are the partners able 
to identify new and creative ways to solve problems? 


By working together, how well are these partners 
able to include the views and priorities of the people 
affected by the partnership's work? 


By working together, how well are these partners 
able to develop goals that are widely understood and 
supported among partners? 


By working together, how well are these partners 
able to identify how different services and 

programmes in the community relate to the problems 
the partnership is trying to address? 


By working together, how well are these partners 
able to respond to the needs and problems of the 
community? 


By working together, how well are your partners able 
to implement strategies that are most likely to work 
in the community? 


By working together, how well are these partners 
able to carry out comprehensive activities that 
connect multiple services, programmes or systems? 


By working together, how well are these partners 
able to communicate clearly to people in the 
community on how the partnerships actions will 
address community's priority problems? 
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zi EE 


Partnerships - continued 


Benefits of participation - for each of the followin 
whether you have or have not received the benefit 
this partnership or network. 


g benefits, please indicate 
as a result of participating in 


Enhanced ability to address an important issue 


Development of new skills —— 

ee 

a i es ieee 

Acquisition of useful knowledge about services, programmes or people in the Wyte: 

community greed 
PRES feo 


Enhanced ability to affect public policy 


Development of valuable relationships ; sabes lapel 
Enhanced ability to meet the needs of your constituency or clients S.-i 


Ability to have a greater impact than you could have on your own 


Ability to make a greater contribution to the community? a ay 
Able to acquire additional financial support? hace dear 


Drawbacks to participating in partnership - please indicate whether you have or 
have not experienced any of the following drawbacks as a result of participating in 
the partnership. 
Diversion of time and resources away from other priorities or obligations 


Insufficient influence in partnership activities 
Viewed negatively due to association with other partners or the partnership 


Have you ever experienced frustration or aggravation as a result of your membership en 
in the partnership? 


Have you ever experienced insufficient credit given to yourself for contributing to the 
accomplishments of the partnership? 
Have you ever experienced a conflict between your job and the partnerships work? 
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Feedback Workshop Reports 


CFA Evaluation - RD Congo 
Rapport de I'atelier de restitution de Bukavu 


1 Introduction 


L'atelier de restitution a été organisé dans l’enceinte de BDOM-Bukavu réunissant tous 
les partenaires de CORDAID et ICCO évoluant au Nord et Sud - Kivu. 


Malgreé l’insécurité et l’incertitude de la tenue de cette rencontre due aux horaires 


incertains des voyages chaque partenaire a fait de tout son mieux pour participer a cet 
atelier (voir liste de présence en annexe). 


Suivant l’expérience vécue a Kinshasa, au lieu d’une journée nous l'avons étendue en 
deux journées et demi. Cela suffisait pour donner assez de temps aux explications, 
ajoutes, échanges, et réflexions. Cela a permis aux participants en provenance de Beni 
et Aru dont les horaires des voyages sont trés irréguliers de participer le deuxiéme 
jour et l’avant-midi du 3*™* jour. Ainsi on a réalisé l’atelier a duré 2 jours et demi (voir 
programme en annexe). 


2 Organisation 


Toute l’organisation matérielle était confiée au BDOM Bukavu qui a offert ses locaux 
pour la tenue de I’atelier. 


3 Déroulement de I'atelier 


Au par avant l’équipe composée de Bonani et Bingo, s’est assurée de la réception du 
document/draft envoyé quelques jours avant a toutes les organisations évaluées ; pour 
ceux qui n’en avaient pas, les services du BDOM/Bukavu nous ont permis de les leur 
multiplier. Contrairement a l’atelier de Kinshasa, ou l’on a fait des exposés, a Bukavu on 
a procédé a l’examen du texte qui était sous les yeux des participants. 


La programmation initiale avait prévu de passer en revue le travail fait au BDOM et au 
Centre Olame, étant donné que ces deux organisations présentent des aspects de travail 
trés complexes. Il fallait mieux les comprendre afin de présenter un travail plus ou 
moins complet. 


, 


Ainsi au compte du BDOM une lecture assidue du texte par les participants a donne 
lieu a des compléments d'information, a des corrections des données quantitntives et 
qualitatives. Les recommandations étaient faites de deux manieres : celles de l’equipe 
d’évaluation et celles formulées par l’institution évaluée a l’endroit du bailleur. 


‘Le Centre Olame quant a lui s’est plu a porter des amendements au texte au niveau de 
la présentation de l’institution, des résultats et quelques commentaires formulés par 


l’équipe d’évaluation. 
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Le BOAD qui lors de la mission de collecte des données était hésitante, a largement 
contribué lors de cet atelier de restitution. En fait, non seulement il a bien accueilli 
le travail restitué, mais avec les autres partenaires il a enrichi le document en y 


s correctifs nécessaires. Quant a l’association ELIMU, qui ne s‘etait pas 


rtant de 
ee s données, la revue 


fait recherché comme ce fut le cas lors de la mission de collecte de | 
du texte a efficacement contribué a reexpliquer certains points qui ne semblaient pas 
indiquer clairement et fidelement la situation de cette organisation au niveau de son 
opérationnalite fonctionnelle, dans un milieu trouble et la coherence de ces activites 
dans une zone en en proie aux problémes posés par la guerre et l’insecurite. 


Le CME avait quelque peu reformulé la présentation au niveau de sa mission, de son 
organisation fonctionnelle. Au niveau de l’impact et de la qualité de ses services, des 
ajouts nécessaires ont été formulés pour mieux rendre son efficacité sur terrain. 


L'IPASC, aprés avoir beaucoup hésité pour raison de transport aerien non sur, est venu 
4 l’atelier. La participation était appréciable dans la mesure ou elle nous a donne plus 
de lumiére sur ses structures qui semblent unique. Quelques éclaircissements ont été 
faits au sujet des stratégies d’intervention sur terrain et des actions communautaires 


développées. 


A la fin des échanges, il a été jugé nécessaire, d’avoir une derniére rencontre de mise en 
commun ou les participants ont eu 4 mieux se connaitre. C’est ainsi que le 30/12/2004 
tous les participants étaient réunies de nouveau avec une bréve présence de Maria 
Masson, la Directrice du BDOM. Au cours de cette séance on a procédé a |’échange 
d’information pratique, des documents et des adresses. Aprés toutes les discussions on 
a formulé le souhait de se communiquer régulierement et de demander aux bailleurs 

de leurs faciliter le plaidoyer au niveau de la Hollande pour la fin de la guerre a |’Est 
de la RDC. A l’issue de cette rencontre une visite a été effectuée a l’usine de fabrication 
des emballages de produits pharmaceutiques. La cons€quence de cette visite est que le 
CME avait décidé de s’approvisionner auprés du BDOM au lieu de continuer a le faire 

a Nairobi ot ¢a coiate plus cher. Par ailleurs grace a l’atelier nous avons pensé qu’il y a 
possibilité d’établir un partenariat entre 1’IPASC et le l"ESP dont les activités semblent 
tres complémentaires. C’est sur cette note particuliére qu’a pris fin notre atelier de 
restitution de Bukavu en date du 30/12/2004. 

Sur le chemin de retour l’équipe a eu une séance de travail avec le Staff du BOAD. 


Conclusion 


En dépit des résultats positifs réalisés, tous les participants a l’atelier, ont émis le souhait 
de voir ICCO et CORDAID peser de tout leur poids auprés du Gouvernement hollandais 
afin de le convaincre de pouvoir s’impliquer davantage dans la résolution du conflit 
dans les Grands lacs aux cétés des autres pays européens, américains et les Nations 
Unies pour revenir la paix en Rdc. Carr sans la paix tout effort de développement dans 
notre pays n’aura pas d’impact. 


Fait a Bukavu le 31/12/2004 
Pour l’Equipe d’évaluation, 


BONANI BUKAKA 


Coordonnateur 
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CFA Evaluation - RD Congo 
Rapport de I’atelier de restitution de Kinshasa 


1 Introduction 


Conformément au programme établi, un 
novembre 2004 réunissant d'une part l’é 
internationale Janneke Roos, et d’autre 
partenaires CFAs. 


premier atelier de restitution a eu lieu le 18 
quipe d’évaluation appuyee par la Consultante 
part les responsables des organisations 


Compte tenu du fait que ICCO n’appuie aucune organisation basée a l'Ouest de la RDC 
l’equipe a jugé pertinent de ne présenter en priorité que les résultats préliminaires faites 
aupres des partenaires financés par CORDAID évoluant a Kinshasa et Kananga, méme si 
une presentation sommaire avait été également faite sur le BDOM Bukavu. | 


Etaient présents, les responsables du BDOM Kinshasa, de l’Ecole de Santé Publique, du 
CONAFED, du SPS et du BDOM Kananga (voir liste de présence en annexe). 


2 Deroulement des travaux 


Assistée par le Bureau de liaison de CORDAID Kinshasa, la mission d’évaluation a été 
effectuée pour collecte des données dans les trois zones concernées et conduite par deux 
équipes réparties de la maniére suivante : 


e L’equipe A comprenant deux évaluateurs (BINGO et BAKATUVUIDI) a sillonné deux 
provinces de 1’Est, le Nord et le Sud Kivu. 

e L’équipe B composée de 3 personnes (Mme MPUTU, MIFUNDU et BONANI, 
Coordonnateur de l’équipe) s’est déployée 4 Kananga et Kinshasa. 


Pour susciter des réactions appropriées et efficientes au cours des échanges et 
discussions, il a été décidé de présenter les données récoltées organisation par 
organisation. Ainsi chaque organisation était invitée a réagir pour rectifier, préciser et 
apprécier les observations de l’équipe d’évaluation. 


Pour donner une idée claire de cette évaluation aux Partenaires de CORDAID, le 
Coordonnateur de l’équipe d’évaluation a introduit la séance en présentant les points 
ci-apres : 


le contexte de l’évaluation 

les objectifs et cibles des CFAs 

les stratégies d’intervention des CFAs 

une bréve présentation de CORDAID et de ICCO 

la méthodologie de l’évaluation 
l’analyse des résultats des recherches faites auprés des Partenaires. 


Aprés cette introduction, les membres de l’équipe ont présenté tour a tour les résultats 
des organisations ci-apres: 
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BDOM Kinshasa 

BDOM Kananga 
CONAFED 

Ecole de Santé Publique 
SPS 

BDOM Bukavu 


Quoique le BDOM Bukavu évolue a |'Est, la présentation de l’analyse des résultats de 
cette organisation était destinée a la comparaison nécessaire entre les organisations 
évoluant dans le territoire en proie aux conflits armés et celles exergant dans les zones 


plus paisibles. 


L’équipe a pu, dans la mesure du possible, presenter quelques expériences du BDOM 
Bukavu aux autres membres a l’atelier (mode de tarification des soins de sante, 
mutuelles de santé, approche contractuelle, etc.). 


Au cours des discussions qui ont suivi les exposés, quelques questions importantes 
avaient retenu l’attention de |’atelier. 


e Au niveau de BDOM Kinshasa , on avait beaucoup échangé sur 2 problemes 
essentiels : 


1° L’'H6pital St Joseph et le probleme d’offre des services percu comme étant des services 
spécialisés. Cette perception n’a pas rencontré l’avis de l’atelier appuyé par le point 
de vu du responsable de 1’Ecole de Santé Publique qui considére que certaines 
pathologies rares de l’ORL ou de l’ophtalmologie deviennent des cas de pathologies 
courantes a intégrer dans la prise en charge générale des soins de santé primaire. 


° 


2° Le probléme de |’insuffisance du personnel surtout médical est resté sans suite et 
nécessite d’étre résolu en priorité. 

Le Partenariat manqué entre le SPS et CORDAID devrait étre relancé. 

Le probleme de plaidoyer au niveau de BDOM pour le renforcement de la Société 


civile a été évoque. 


Apres discussion, des amendements ont été apportés au document de présentation. 
Pendant les différentes interventions, il avait été également donné des précisions de la 
part de représentation de CORDAID Kinshasa pour élucider certaines questions entourées 
des zones d’ombres. 


Le probléme de SPS a été longuement discuté entre le BDOM, CORDAID et l’équipe 
d’évaluation. Il s’en est dégagé l’option de recommander a ce service la réalisation 


effective de sa mission de coordination des activités de l'ensemble des BDOMs a travers 
la RD Congo. 


Pour ce faire, il a été recommandé au CORDAID d’entrevoir la possibilité de relancer son 
appui a l’endroit de cette structure par le renforcement de sa capacité institutionnelle. 


Quant au CONAFED, a l'exception de son objectif qui aurait été mal défini, il avait 
maintenu les résultats du travail présenté par les évaluateurs. 
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Le BDOM Kananga était représenté 


WO Par son Directeur qui se trou 
formation a I'Ecole de Santé Publiq q ve actuellement en 


ue a Kinshasa. Selon les résultats de l’évaluation 


an). La problématique du payeme 
souleve. 


L'exposé sur le BDOM Bukavu a permis au BDOM 
des legons a apprendre, spécialement en matiére d 
contractuelle et autres. 


Kinshasa de mettre en évidence 
‘auto-financement, d’approche 


Concernant la réduction de la pauvreté, il a été remar 
beaucoup, en réalité l’appui financier qu’on donne ne 
processus, pour donner la chance a la réduction de la 
devrait se faire. 


qué que pendant qu’on en parle 
fait que diminuer. En tant que 
pauvrete, c’est le contraire qui 


3 Conclusion 


Selon les observations faites par les participants au cours de cette séance de restitution, 
on a retenu ce qui suit : 


¢ que l’approche de l’appui partant des droits au lieu des besoins, l’approche genre, la 
santé sexuelle et de la reproduction liée a la lutte contre le VIH/SIDA soit encouragée 
et bien reformulée ; 

¢ qu'il soit recommandé auprés des organisations |’établissement des statistiques et la 
collecte des statistiques sur la femme ; 

© que le 1™ rapport (draft) soit envoyé a toutes les organisations pour qu’elles en 
connaissent la teneur afin d’éviter d’éventuelles divergences. 


Fait a Kinshasa, le 18/11/2004 
Pour l’équipe d’évaluation 


BONANI BUKAKA 
Coordonnateur 
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Q1 What data you tried to collect from the different organisations you visited? 


Réponse 1 


Les données collectées étaient quantitatives et qualitatives. Les données quantitatives 
devraient nous montrer l’ampleur mesurable des phénomeénes qui ont subi des 
changements (positif ou négatif), par exemple le cas des maladies sélectionnees, 


mortalité, etc. 


Quant aux données qualitatives, elles devraient servir a démontrer un changement 
difficilement mesurable vu le caractére insaisissable du phénomene. C’est donc des 
appréciations descriptives des personnes interrogées mais dont les réponses peuvent 
changer selon les circonstances. C’est le cas, par exemple, des donnees relatives a la 
prise de décisions au sein des organisations, de la synergie entre les personnes et les 
organisations au sein du partenariat. 


En fait toutes ces données sont en relation avec les termes de référence auxquels étaient 
attachés quatre sortes de questionnaires indiquant la méthodologie organisationnelle 
destinée aux groupes cibles ci-apres : 


mandataires de |’Etat ; 

responsables et personnel des organisations évaluées ; 
groupes intermédiaires (personnel des services de santé) ; 
bénéficiaires. 


Q.2 How you collected the community members/beneficiaries for interview, was it 
opportunistic (just talking to whoever was around) or were you more purposeful in your 
selection ? What worked and what didn't ? 


Réponse 2 


Avant tout il fallait comprendre l’organisation de la structure a évaluer du sommet a 

la base. Et apres cette étape on pouvait planifier une procédure d’échantillonnage. Par 
exemple, pour les organisations partenaires, les responsables du sommet étaient d’office 
interviewés en vue d’obtenir l’image et la vision claire de leurs institutions. 

Cela est de méme pour les mandataires de l’Etat a différents échelons au niveau du 
ministere de la santé et au niveau provincial (c’est le cas de ...). 


Pour les hépitaux, les centres médicaux et les centres de santé, on a interrogé les 
directeurs d’abord. Ensuite on est descendu au niveau des services ot on a eu des focus 
groups avec les chefs des services. 


Dans les centres des jeunes, notre visite devrait coincider avec la journée de réunion 
en vue d’obtenir des points de vue plus diversifiés sur leur engagement au probléme de 


lutte contre le VIH/SIDA, par exemple. 


Pour les bénéficiaires 


Au niveau de I’hépital, on a sélectionné les malades internés suivant la durée de temps 
passée a l’hdpital, selon le sexe, selon l’age, selon le statut social et la charge familiale. 
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Tous ces criteres ont ete définis et suivis Par souci d’obtenir des informations plus ou 
moins diversifiées fables, et méme vérifiables. 


Ensuite on était intéressé par les gardes malades 


qui sont souvent des femmes pour 
essayer de comparer leurs réponses 4 celles donn 


ees par les malades. 


Quant aux malades soignés en ambulatoires ils 
dans la salle d’attente tout en respectant la 
possible. 


ne pouvaient étre pris que au hasard 
parite homme-femme lorsque cela était 


Dans tous les cas, il n’y a pas eu une forme de choix 
qu'une autre du fait dans le cas de ] 
malades ont vivement apprécié la q 


eae qui a donné plus de satisfaction 
hopital, par exemple, les malades et les gardes 


: ualité de soins recus mais par contre ils ne sont pas 
d’accord avec le tarif leur appliqué. De leur cété les responsables des hépitaux affirment 


que leurs tarifs sont tres bas par rapport aux autres formations médicales privées et de 
)’Etat. 

En conclusion, les résultats obtenus semblent trés satisfaisants dans la mesure ot aprés 
avoir visité en privé un centre hospitalier et un centre de jeunes a Kinshasa spécialement 
en guise de contrdle d’attitude, aucun changement de reponse n’a été enregistré. 


Cependant, notons que ce qui n’a pas marché c’est le fait que les gens manquent 
d'information assez souvent. 

Ainsi au cours de cette recherche les bénéficiaires répondant a nos questions, ils ont 
toujours émis le doute quant au changement de la situation en leur faveur. Ce qui 
explique de temps en temps des réponses imprécises 4 quelques questions. 
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Agence de Co-financement CORDAID, ICCO, PLAN Netherlands 


Evaluation des Partenaires 


Programme d'atelier de restitution de Kinshasa 


Jeudi le 18 Novembre 2004 


09H00’ - 09H10' : Introduction de l’Atelier (Coordonnateur de 1’équipe) 

09H10’ - 09H40’ : Présentation des résultats de la mission 

09H40’ - 10H10' : Commentaires sur les résultats de la mission de Kinshasa 

10H10’ - 10H40’ : Présentation des résultats de la mission de Kananga 

10H40’ - 11H10’: Commentaires sur les résultats de la mission de Kananga 

11H10’ - 11H30’: Pause café 

11H30’ - 12HOO’ : Présentation des résultats de la mission de l'Est : BDOM Bukavu 
12HOO’ - 12H30’: Commentaires sur les résultats de la mission de 1’Est 

12H30’ - 13HOO’ : Recommandations 


13HOO’ : Repas et cléture 
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Agence de Co-financement CORDAID, ICCO, PLAN Netherlands 


Evaluation des Partenaires 


Programme d'atelier de restitution de Kinshasa 


28 - 29/12/2004 


Le 28/12/2004 


09HOO’ - 10HO0O’ : 
10HOO’ - 10H20’ : 
10H20’ - 10H30’ : 
10H50’ - 12HO0’ : 
12HOO’ - 13HOO’ : 
13HOO’ - 15HOO’ : 
15HOO’ - 15H30’ : 
15H30’ — 16HOO’ : 


Le 29/12/2004 


09HO00’ - 09H30’ : 
10H30’ - 12HOO’ : 
12HOO’ - 13HOO’ : 
13HOO’ - 14H30’ : 
14H30’ - 15H30’ : 
15H30’ - 15H40’ : 
15H40’ - 17H30’ : 


Préparation matérielle (BDOM) 

Introduction (Coordonnateur de l’équipe) 

Echange sur les objectifs et la méthodologie de l’évaluation 
Présentation du BDOM Bukavu (1** partie) 

Repas 

Présentation des résultats du BDOM (2*™° partie) 

Pause café 

Introduction présentation des résultats du centre Olame 
(1° partie) 


Presentation des résultats du centre Olame (2*"* partie) 
Présentation résultats du BOAD 

Repas 

Présentation des résultats de l’association Elimu 
Présentation des résultats de CME/Nyankunde 

Pause café 

Présentation des résultats de l’IPASC 


Une demi-journée supplémentaire 


Le 30/12/2004 


09H30’ - 11HOO’ : 


11HOO’ - 11H45’: 


12HOO’ : 


Discussion générale (échanges entre organisations pour une 
connaissance mutuelle) 

Visite de l’usine de fabrication d’emballage des produits 
pharmaceutiques du BDOM Bukavu 

Fin des travaux. 
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Liste de presence 
A Iatelier de restitution de Bukavu 


SHAMAVU Désiré Centre Olame Secrétaire 98623616 centre_olame@yahoo.fr 
Administratif 


MUSHAGALUSA B.D.0.M/Bukavu 97733886 bdombkv@yahoo.fr 
Charles charlesmushaya@hotmail.com 
03 KAWA LUSENGE BOAD-Goma 98624002 boad@rwanda1.com 
Frédéric boad01 @yahoo.fr 
freddykawa @yahoo.fr 
04 WACHAN KIKA CME Nyankunde Coordination des | 0813026654 cmenyan @tiscali.fr 
DJAYEROMBE projets 


ATSIDRI ASSIA Elias | IPASC Représentant 0810102383 ipasc@bushnet.net 
Légal / Secrétaire | 0810252650 
Académique a.i 


NTEBA MAKUMBA | Ass. Elimu/Uvira Coordinateur 98674684 elimu_uvira1 @hotmail.com 
Jules julenteba@yahoo.fr 
BONANI BUKAKA | IHSD 0816901615 _| bbukaka@yahoo.fr 
08 | BINGO KIMPWENE | IHSD 9905099 belvisza@yahoo fr 


Responsable 
Projets Santé/ 
Coordination 

Médicale 


Administrateur 
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Liste de presence 
A l'atelier de restitution de Kinshasa 


Le 18 novembre 2004 


KAJANGU Jean- ESP Secrétaire du 0816676613 jkajangu@yahoo.fr 
Pierre programme 


03 | Dr. MINZEMBA —_| BDOM/Kinshasa Médecin BDOM —_| 98231693 bdomkinshasarde@ yahoo fr 


Dr. BENEDICTE BDOM/Kinshasa Medecin Directrice | 98231693 bdomkinshasarde@yahoo.fr 
CLAUS 
Dr. KANOWA BDOM/Kananga Meédecin Etudiant | 0816041148 bdomkananga@yahoo.fr 
MUNUNGAWI a l'ESP 
NKOY Hypolite SPS Chargé des 
programmes 
07 MUKOMBA Roger | CONAFED Administrateur & | 98378647 conafed @ic.cd 
Financier 
ANNIE LEFEVRE CORDAID Représentant 0815088484 alefevre @ic.cd 
CORDAID 8975256 
CORDAID Consultant 0817005545 wetshi@hotmail.com 
OSOMBA 
JANNEKE ROOS IHSD Consultante roosjanneke@hotmail.com 
Internationale 
Dr. BONANI Consultant 0816901615 bbukaka@yahoo.fr 
BUKAKA National 
Dr. MIFUNDU IHSD Consultant 0814064932 mifundu@hotmail.com 
BILONGO National 
MPUTU DIERI IHSD Consultant 98269479 d_mputu@yahoo.fr 
Monique National 


BINGO KIMPWENE | IHSD Consultant 9905099 elvisza@yahoo.fr 
National 

BAKUTUVUIDI IHSD Consultant 0818144750 makanibak@yahoo.fr 
National 8944608 


Téléphone 


munyangam @yahoo.fr 


Dr. MUNYANGA 
MUKUNGO 


Directeur adjoint 9942798 
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NEPAL feedback workshop 


The Nepal feedback workshop was held on the 22nd of December. All partner 
organisations that participated in the evaluation were represented at the workshop. 


These included : 


BNMT 
CBRS 
CCDN 
CVICT 
Plan Nepal 
SSS 
WOREC 
YUHP 


The evaluation team presented their findings to partners, who then commented on 

the results. The discussion centred primarily on the fact that those attending the 
workshop agreed with the findings of the evaluation team and felt they had little to 
add to the discussion. The only exception to this was clarifications provided by both 
BNMT and Plan on aspects of institutionalisation of their programmes into government 
programmes, and issues around linkages with local government. 


The following slides are those that framed the feedback to workshop participants. The 
workshop was held in Nepalese. 


Presentation Outline 


CFA Health Evaluation: NEPAL 


e Presentation of Preliminary Findings 
- Process/Methodology and Limitations 
— General Overview 
- Organizational Issues 

i Riccio wed Chnanioany Cake - Issues for further exploration 


e Open Discussion and Feedback 


Process/Methodology 


Initial Workshop (12 Oct) 
Documents Review 
Organizational Interviews (Group and Individual) With Partners and Line 


oa Group Meetings (Mothers Group, CBOs, Users Group, Volunteers 
Field Visits: 

- Makwanpur (CCDN) 

- Kaski /Syngja (CBRS) 

Bara (PLAN) 

- Morang (PLAN, and BNMT) 

Organization Development and Partnership Assessment 
® SWOT Analysis at Organizational Level 

2"° Round of interviews (As per Need) 

Debriefing (21" Dec) 


Limitations 


e Limited time for field visit caused limited 
interaction with community groups 


e Diversity of Program Types 
e Diverse Nature of the Organizations 
e Lack of Base line Data 
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General Overview nalovence with National Goal/Priority 


Relevance with National Goal/Priority PRSP 
Inline with ‘Health For Ail : , 
re ntact Government- (but not pre they bencowryenpen NGOs)” 
Inline wi ; ; 
Po een eee - @ with Government's Priority for Primary Health Services 


omotes ‘Right Based Approach’ for Health and Nut 
Conflict (PRSP) in Service Delivery but Not in oe = 


Partnership Management: Relationship issues with Indirect Contribution to the Liveliho 


Coordination/Networkin Pro Poor Focus and Involvement of Commu 
Users perspective ade epee 's Approached by the Programs ne 
ack of (inconsistency) Baseline and Quantitativ D 
Challenges Ahead difficulty in Attribution of the Tamel emai, - oraatas 
Level Indicators 


Targeting (Access/Use/Control) Organizational Dynamics 


Improves Access to, and Use of Basic Health Governance/Leadership/Management Structure Varies 
Services of the Marginalized and Vulnerable groups INGOs and Some NGOs Have Strong Management ' 
Most of the Programs Encompass Social Inclusion Systems — 

Issues (Dalit, women, Children, Poor and 


Marginalized) Structure of the majority Partner NGOs: 


° CRO Beeed Management (Control) Critical Thinking about the ‘Staff NGO Model’ to avoid 
e ‘Control’ Varies According to the Program Approach conflict of Interest is emerging among the NGOs 


e ‘Feel In the Gap’ Program wise and Time wise **) Professional Staff are engaged by all Partners 


Overlap Between Governance and Management 


Partnership Management: Relationship 


issues with ICCO/NLNO (PLAN) Coordination/Networking 


Partners emphasize linkage and coordination with HMG line 
e Stable Resource Su Pport agencies, mostly work in close coordination/collaborations 


e Responsive but Less Interfering Institutionalization of the work within HMG system is not very strong 
— . ee . at Meso level (Some Exceptions YUHP, CCDN): 
e Limited Direct Monitoring Interaction Creates Vision and Commitment for institutionalization within HMG System is 


Confusions Sometime limited among most of the field staffs, 


tion of rks are strong (ECCD, 
e Less Support For OD of NGOs Bw haan mane ion of some works are strong 


e Flexibility in Budgeting and Use of Fund is No networking and coordination among CFA (mainly ICCO) 


Enabling to Respond to the Needs of the Partners in the country 
Community HMGI/LA are mostly positive and cooperative to the Partner 
NGOs/INGOs-Varies on Individuals at the LAs 


Users perspective (Impact) Conflict 


Health Seeking Behavior Improved Among the Target People: Partners Are Not Targeted Directly 
(Increased Immunization, MCH/RPH, personal health and hygiene No Direct Threat To The Programs 


and nutritional practices among the users) : 
Improved Overall Enrollment (girls enrolment also) and Improved Delay In Implementation Is Caused Because of 


Schooling Condition Disturbances 

Improved Demand Making Capacity (CBO) More Transparency Approach Is Adopted 

seach Vereeipen and Janered Weoeer Direct Involvement Of Community Helps 

nt et EEE os 4) Small NGOs have Less Sensitivity on Staff Security 
Improved Confidence level of Target Groups (esp Women) 

IG contributes in Livelihood Improvement of the Target Group 


Added Value By CFA Support 


Community mobilization resulting in realization of 
health needs, making demands for, increased 
use of health services, and community control of 
service system 

Integrated Program Approach adopted by many 
partners helps the beneficiaries not only in good 
health but also in increased life quality at family 
level 


Organizational Impression 


e CBRS 

e YUHP 

e PLAN 

e CCDN 

e WOREC 
e SSS 

e CVICT 


e@ Strong Community Connections and Community 
Ownership 
e Serving poor and disadvantaged groups 


e Health and Livelihood is both addressed by the 
programs 


Good Relation with Line Agencies 

Able to work in difficult situation 

Longer Term Vision should be developed 
Organizational restructuring issue is emerging 
Phase-out strategy is not defined clearly 


Yala Urban Health Program 


Institutionalization Process is very strong, 


Mainly Serves Poor, Marginalized women and Children 
in urban areas, 


Addresses Critical and unmet Need of the Target Area 
(RPH, MCH) improving Access to and Use of Basic 
Services 


Good Community Ownership and Participation 
Service range are very nominal compared to the need 
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Challenges Ahead 


Small NGOs more dependent on Single Donor 

‘UMN Image’ Pros and Cons of It: Building Independent 
Image 
Developing Longer Term Intervention Vision: Exploring 
New Opportunities 

Specialization and Sustainability of ‘Core’ Organization 
Research and Documentation of Emerging Models and 
Best Practices 

Advocacy For Health Rights 

Developing Institutional Collaborative Situation with LA 
and Local Bodies 


Governance Model Is very Impressive and Dynamic 
Good Technical Command 

Efficient Management, Dedicated Staff Team 

Scattered Approach 

Lack Longer Term Sustainability Vision For The Services 
Less Linked With Community and Local Institutions 


Service Spectrum And ‘Phase in Phase Out’ Strategy Should be 
Worked-out 


Target Achievement is Positive 
Board Should Become Proactive in Marketing of The Organization 


Advocacy for the Rights of Differently Able People Should be 
Considered 


Sakriya Sewa Samaj 


Specialized Service Linked with National Health Priority 
Agenda (HIV/AIDS), 


Some collaboration with Government Agency 


Resource Diversification Should be considered as 
longer term plan 


Overlap Between Leadership and Operational 
management level 

‘Positioning’ and ‘finding niche’ is a challenge in 
growing competition in the sector 


Community based, women focused empowerment 
program well accepted by the users, 


Strong CBOs are emerging out of WOREC groups 


RH, |G and Community Organization is main intervention 
area 


Not dependent with one Donor 
Local Capacity Building 
No specific health focus-lack health indicators 


Sustainability is a challenge although community 
involvement is good 
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Focused Specific Target Group 
‘Longer Term funding’ is helpful for the organization 


Addresses Contextual Need in a Supply Led 
Situation 


Sustainability of the service will remain as challenge 


Service Process and content with proper discharge 
strategy should be worked-out in detail 


Proper Recording and tracer survey for Impact 
Monitoring should be included in the program 


ood Partnership with Government 
Long commitment and contribution in the area 


Working for ‘Government’ (DOTS, CDP) Approach 
raises the question of sustainability 


Local partnership and community based approach 


Compliments service gap in health sector in the 
region 
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PLAN-NEPAL 


ECCD have become core competency of PLAN 
Good linkage and contribution at Macro Level for ECCD and IMCI, 


Good nity j i 
ECCD seme ity nvolvement and ownership in both health and 


Multi Actors Model of Morang is a good each 
perfection and replication ° sam one 
Inconsistent approach within PLAN in the ECCD models 


Meso Level Linkages are less institutionali Spite 
collaborations with Line Agencies, —€ . 


There are Gaps at the lic P. ' 
coninas ps Policy level, PLAN has potential to 


Action Research, Documentation of Best Practices and Advocacy 
for minimum standards In both ECCD and Health programs 


Issues for Further Exploration 


e HMG Perspective 
e CFA Perspective 
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Zambia feedback workshop 


Scope of the study 


This study involved a number of Catholic Dioceses in Zambia. These include Mansa 
Diocese, specifically targeted at Nchelenge and Lubwe. Mpika Diocese covered Mpika 
area. The Diocese of Ndola was also part of the research as well as Lusaka focused at 
Civil Society for Poverty Reduction (CSPR) and the Catholic Center for Justice, Peace 
and Development - CCJPD. PLAN Zambia was also included, focusing mainly Mazabuka 


and Chibombo. 


eel Objectives of the study 
The objectives of the research included: 


i To find out which visions, strategies and activities partner organisations have to 
improve access to, use of and control over basic health services by potential users. 

ii To find out the qualitative and quantitative results of the health programmes and for 
whom they improve access, use of and control over basic health-related services. 

iii To find out the extent complementary partnerships, participation in networks and/or 
alliances contribute to better performance partners. 

iv To find/establish demonstrable results or outcomes from partnerships working in 
terms of performance of partner organisations and impact on the health system in 
general. 

v_ To highlight the added value by Cordaid and PLAN Netherlands’ support for the 
partner organisations and their programs. 


1.2 Rationale of the study 

The major objective of the research was to gain insight into the contribution of 
Cordaid and Plan Netherlands and their partner Organisations to the improvements 
of sustainable use of, access to and control over health-related services for the poorer 
sections of the population, with a special emphasis on women and youth. 


t3 Operational definitions 

CHAZ Churches Health Association of Zambia 

CCJPD Catholic Centre for Justice, Peace and Development 
CDEs Classified Daily Employees 

CHNC Community Health Neighbourhood Committees 
CELE Church Health Institutions 

CSPR Civil Society for Poverty Reduction 

DHMTs District Health Management Teams 

FGDs Focus Group Discussions 

H.B.C. Home-based Care 

PMTC Project Management and Training Consultancy 
RDCs Resident Development Committees 

TBA Traditional Birth Attendants 


ZAMSIF Zambia Social Investment Fund 
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Methodology 


Different data collection methods were use 
Focus Group Discussions ~ FGDs, with beneficiaries of the projects under Cordaid 
conversations with key people in the areas Visited in health centres and staff pune the 


District Health Management Teams - DHMTs Sec 
- Secondary data , 
of available documentations in relevant fields. . was also used, in the form 


d in this research. These included 


2.1 Sampling procedures 

For this part of the report, the Sampling of the partici 
was in order to have right people come up with ri 
research. 


Pants was done purposely. This 
ght information for the purpose of the 


These participants were mainly to react to the report provided by the Prevention of 
Maternal Death - PMD, who were contracted by Project Management and Training 


Consultancy - PMTC. The report analysis was conducted at Pearl Haven Inn, in Lusaka’s 
Rhodes Park area. 


The participants were: 

1 Mr. Kabwisa Liywalii - Coordinator, Prevention of Maternal Death — PMD 
2 Mr. Patrick Kunga - Prevention of Maternal Death - PMD 

3 Mr. Chanda Fikansa - Ndola Diocese 

4 Mrs. Priscilla Phiri Bweupe - Chipata Diocese 

5 Father Chansa - Mansa Diocese 


22 Problems encountered 
The research team that carried out the initial report met a number of limitations in their 
work among them are the main ones: 


a The fieldwork was conducted long after it was initially planned for. This resulted in 
a number of key informants not being found in their office. In other cases, it was 
difficult to secure appointments for interviews with target respondents as some of 
them were out on Christmas break. 

b The limited number of days per site led to weekends being used for interviews a fact, 
which proved difficult as respondents came from Institutions that do not work during 
this time. 

c In some cases, staff from government institutions proved difficult to provide 
information related to the evaluation. 


Research findings and analysis 


3.1 Introduction - 
This part of the report is mainly centreed on the analysis provided by the participants, 
who were reacting to the report provided by Mr. Kabwisa Liywalii - Coordinator of 
Prevention of Maternal Death - PMD, after the fieldwork. In its approach, the report 
will provide firstly the findings of PMD and then the reactions and comments of : 

‘the participants in as far as their ground experience is concermed in their respective 


Dioceses. 
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ers' views on the impact of Cordaid Aid Projects 


ent stakehold 
su pai ncerning the 


The government stakeholders interviewed provided various responses CO 
impacts of the Cordaid and PLAN Zambia sponsored health projects. 


For example, they acknowledged that these projects were making meaningful impacts in 
their respective Catchment Areas. 


They also noted that the quality of health service delivery by partner organisations in 
the districts where government stakeholders were interviewed was reportedly high. 


Further, beneficiary involvement in health services planning and delivery varied, and 
depended on how individual managements link with the community. 


On the other hand, it was reported that there was poor information flow between partner 
organisations and the District Health Management Teams - DHMTs. 


In reacting to these findings, the participants who analyzed the initial PMD report 
expressed their happiness over the DHMTs’ acknowledgement to the high quality of 
health care from Church Health Institutions - C.H_Is. 


However, the participants expressed a number of concerns in as far as DHMTs were 
concerned. For example, it was revealed that DHMTs still had a mentality that the 
Roman Catholic Church gets a lot of financial resources from rich missionaries to 
fund the Parishes. This mentality was reportedly contributing to the poor funding of 
government to C.H.Is. 


It was actually perceived that DHMTs were biased against C.H.Is. An example was cited 
of Lubwe Mission Hospital, which had a bed capacity of One Hundred and Sixteen (116) 
and Samfya District Hospital, a government run. The latter was reportedly highly funded 
as compared to the former on the basis that the former was a Church run Institution, 
with a perception that the Roman Catholic Church had enough money to fund her own 
hospitals. 


The other problem noted was that of lack of information flow between DHMTs and 
C.H.Is. This was attributed to the fact that since the Church does not receive funding 
from government, there was no need to be asked for information and reports on projects 
it (government) does not fund. As such, the Church was allegedly less transparent in the 
revelation of what actually goes on at ground level, as government, through the DHMTs 
was not appreciative of the contributions of C.H.Is in health care provisions. 


In as much as the calls for transparency were all right, it was noted that C.H.Is do not 
win, as they get no money from government, especially after revealing their resources 
at hand. This makes the government feel that after all; the C.H.Is can have alternative 
sources of funds \even without government funding. 


This scenario has led to C.H.Is holding on to information required by government. 
Instead, they prefer submitting their reports to Cord AID, which funds them, and to 
Churches Health Association of Zambia - CHAZ- to which they are affiliates. 
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=~ — concern raised by the participants was the lack of funding from government 
owards Home-Based Care projects in C.H.Is, As such, these institutions depended much 
on volunteers to carry out the duties of Home-Based Care giving 


Coupled with this concern was the failure b 
as well as the squabbles that characterised 
case was cited of St. Paul’s Mission Hospit 
of Zambia. It was however revealed that some C.H.Is managed to secure housing 


allowances for their staff. This was attributed o the di arwe 
aaa ifferent policy int 
personnel in different DHMTs over housing allowances. a 


y overnment to second nurses to C.H.Is, 
the disbursements of housing allowances. A 
al in Nchelenge District of Luapula Province 


It was thus stressed that the caliber of personnel heading these DHMTs also mattered 
The participants noted with concern that the ignorance expressed by some DHMT | 
Managers over health matters left much to be desired, and were contributing to the 
current perceived misunderstandings between government and C.H]Is. 


When it came to seconded nurses to C.H.Is, Churches were left to pay them on their 
own. For example, in Ndola Diocese, it was reported that out of the forty (40) nurses, 
only five (5) were sponsored and seconded by government, while the church sponsored 
the remaining Thirty —Five (35). 


This claim was substantiated by one of the participants, Mrs. Priscilla Phiri Bweupe, a 
nurse by profession, seconded to a C.H.I in Chipata Diocese, and has since been removed 
from government pay roll, with a notion that the church would be paying her. 


At this point, a question was raised as to whether CHAZ could play any role in trying 

to resolve this anomaly. However, it was pointed out that CHAZ could only speak to the 
government at a more general level over the concerns of the churches, but could not get 
to negotiate for individual churches, as the Association does not run these churches. 


Instead, individual churches could take it up and negotiate with government for the 
conditions they feel need attention. 


An example was cited where CHAZ played a general role in the negotiation for the rise 
in government funding to health institutions, from the then 25% to the current 75%. 


As a national advocacy tool concerned with policy issues, that was how far CHAZ could 
go, not to negotiate for individual independent churches over their grievances. 


ae Organisation staff views on impacts of Cordaid projects | . 
In the report given to the participants to analyse and react to, the staff in C.H.Is visited 
by the research team rdised a number of issues. 


For example, the staff alluded to the fact that health services provided by C.H.Is were 
located specially in under-served and hard-to-reach places, thus adding value to the 
local communities. 


‘However, user fees were seen to be a deterrent among the poorest sections in these same 
communities. 
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ern stressed, coupled with low funding from 


as also another conc ! 
a hip between C.H.Is and the DHMTs in some 


government, as well as a weak partners 


cases. 


Reacting to the issue of understaffing in C.H.Is, the participants noted that this was a 
general state of affairs in the entire health field, government hospitals included. It was 
pointed out that some hospitals do not even have a single doctor, with nurses attending 


to all cases, regardless of their complications. 


This understaffing was attributed to the general poor conditions of service in most 
hospitals, both government and C.H.Is. Most nurses and doctors were reportedly running 
to Non-Governmental Organisations - (NGOs) and urban hospitals and health centres. 
Others were also said to have left for the United Kingdom - U.K- in search of so called 


greener pastures. 


Other health practitioners were moved from C.H.Is to government institutions, but 

no replacements were made following such moves, thus compounding the levels of 
understaffing in C.H.Is. In fact, it was also revealed that the government had a tendency 
of looking for its own health workers in the institutions it runs, while leaving out C.H.Is 
as separate entities from DHMTs. 


For example, Ibenga Mission Hospital on the Copperbelt Province of Zambia was 
supposed to have eighty (80) nurses, but only had twenty-six (26); needed eight (8) 
clinical officers, but only had three (3) and needed to have five (5) doctors, but only had 
three (3). 


The participants noted that this situation needs quick intervention from government, 
as it was beyond the capacity of C.H.Is. There was need to improve the conditions of 
service for these health workers if they were to be retained. 


The poor conditions of service also meant that even Post Graduate Health workers were 
not retainable, as they preferred urban areas to rural areas, which provided no much 
incentive both for themselves and their families. It was actually revealed that no matter 
how much money a health worker was getting, it made no difference of they could not 
use that same money on valuable items and services, which are not even there in most 
rural areas. 


Commenting on the perceived unaffordable user fees for some rural settings, it was 
noted that limited funding from the government made the question of user fees 
inevitable. 


Originally, the health services in C.H.Is were free of charge, but today it was impossible 
to offer such services, as they need funds to continue with their service provisions. 


Actually, it was revealed that even those government sponsored health institutions on 
100% funding were still lacking in most of their needs. The money received is still not 
enough to cater for all their needs and demands. 


Another issue that was brought out was the area of infrastructure in most of the C.HIs. 
It was pointed out that government does not fund C.H.Is in areas of infrastructure, as 
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they do not qualify for the Zambia Social | 


; nvestment Fund - ’ 
also no longer funding infrastructure ees ZAMSIF. Cord AID was 


opment anymore. 


ika of Northern Province of Zambia, 


These low levels of funding have led to hospitals like Chilonga in Mpika District to 
reduce the bed capacity, form two hundred and thirty (230) bed spaces to fifty (50) 
just because they could no longer have many people come to the hospital for services 
This has been as a result of the local community failing to meet the cost of user fees | 
realising that most of them only have a one-off kind of income from subsistence | 
farming. This restricts their income sources, Common with most rural areas in the 


country. 


In Chundaponde in the Northern Province of Zambia, the payment of user fees in 
kind was encouraged, but the results were disheartening, as the agricultural produce 
so collected did not find any ready market. The health centres turned out to be on the 
losing side. 


The issue of the failure of Home-Based Care projects in government health institutions 
was also raised. It was concluded that these failed mainly because most of the money 
that was received went towards administrative costs. This was a direct contrast to what 
happens in C.H.Is, where most of the work concerning H.B.C. was done on voluntary 
bases. 


This meant that most of the money went to the health workers as opposed to what went 
to the H.B.C. pogrammes. For example, in Ndola, it was revealed that over Ten Thousand 
HIV/AIDS patients were under H.B.C projects, with over Seventy (70) community-based 
volunteers, working with forty (40) qualified nurses. 


This resulted in most of the funds allocated towards H.B.C [projects being completely 
used as intended, thus proving them more meaningful in C.H.Is than the case is in 
government health institutions. 


3.4 Relations with the relevant Catchment Focus Areas-CFAs wis 
In the report provided, the results showed that respondents were appreciative of the role 
played by their respective CFAs. 


Cord AID was also said to have had undergone a lot of transformation in the recent 
past. The ever-changing focus areas and personnel was another area of concern, as 
well as the tendency by Cord AID to allocate specific percentages of funds to specific 
projects. 


Further, it was suggested that Cord AID needed to appreciate the environment under 
which its partners in Zambia were operating. 


Following these findings as provided in the report, the analysts appreciated the long- 
standing relationship that existed between Cord AID and its partners in Zambia. The 
participants appreciated the fact that Cord AID was also under pressure from the 


176 EvatuaTtion oF HEALTH THeme -— SEPTEMBER-DECEMBER 2004 


ome and their funding partners in the Netherlands to have results, 


back h 
sapiesiedeictF d as possible. 


thus also pushing the Zambian partners to be as result oriente 


However, it was noted that the transformation conducted by Cord AID was somewhat 
rushed, as the operation environment in Zambia was not as favorable as it might be in 
the Netherlands where Cord AID operates from. This forced transformation was alleged 
to have resulted into the loss of sight of the local operations, as the environment did not 


provide for such rushed transformations. 


The participants claimed that Cord AID was not very sure of the capabilities and 
limitations of their Zambian partners in as far as the working environment was 
concerned, or else, they could not have been asking for results from them in so short a 
time. The partners complained that they could not adapt to that rushed way of handling 
issues, a Situation that has often been translated as failure on the part of the partners in 
as far as Cord AID was concerned, thus affecting the funding to such local projects. 


The other area of concern was the delayed funding of some projects by Cord AID. This 
was said to have been retrogressive in that some projects have gone for years without 
being implemented due to lack of funds from Cord AID. 


This was attributed to a number of reasons, one being the lack of a smooth 
communication line between Cord AID and its Zambian Partners. The high staff turn- 
over at Cord AID was also identified as having had contributed to the seeming loss of 
sight of who Cord AID is, leading to the repetition of certain procedures before any 
project from Zambian partners could be approved, as each new person in charge of 
projects asked the partners to provide all the information about their projects even those 
they have already submitted when a different individual was in charge. 


It was also noted that the merger of different organisations to form up Cord AID seems 
to have compromised the efficiency and delivery of health services, as some of those 
people in administration allegedly lack knowledge in health matters. 


The use of Interns for just few months by Cord AID was also another area of concern 
by the Zambian partners. They said this did not create continuity, as individuals were 
replaced just at the time when they were getting used to the system. This meant that 
those who come in the system have to go through the learning and adaptation process 
before they could prove useful to the partners of Cord AID. These Interns were also 
allegedly lacking in background knowledge concerning Cord AID partners in Zambia, 
thus they would not appreciate the hardships they go through. 


Conflict of understanding in as far as projects were concerned by Cord AID staff was 
also noted as another area that needed urgent attention. A specific example was cited 
where in the Chipata Diocese, there was some conflict of understanding over which 
project was to kick off. The Finance Officer was reportedly in favour of a different 
project, while the Programmes Officer also had his own preference. This left the Diocese 
confused as to which project was to be carried out that p[particular moment. As such, 
It meant that whoever was on the ground to be in charge of the project won, and 


called for a fresh start to the whole project. This was time consuming according to the 
partners. 
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just send a lamp sum of funds without 
h area. 


Unfortunately, the partners noted that f they used the mo 
outside the stipulations of Cord AID, the 
they said was not health. 


3.5 Intermediary views 
These mainly included the hospitals, health centres and the staff in these same places. 


They generally were of the view that the facility user fees were fairly affordable, but the 
community members usually complained of unaffordability. 


They also noted that some popular structures of the C.H Is were active, while others were 
passive. However, despite this, community-based health providers and their agents were 
said to be making an impact in their communities. 


With this brief report on the views of the intermediaries, the participants who 

were making analyses to the report noted with dismay that most Copperbelt-based 
Community Health Neighbourhood Committees - CHNC- were politically motivated as 
the case was with most Resident Development Committees- RDCs. This made it difficult 
for the Church to work with such CHNCs due to these clear political aspects. 


The Church preferred to work with neutral members of society ion its health care 
provision, because the influence of the political party in government made the church 
uncomfortable in terms of its operations. 


The participants lamented that the involvement of politics in any health projects was 
always detrimental. The best was to avoid them all together. 


3.6 Beneficiary views 

The beneficiaries of the Cord AID sponsored health projects confessed that these 
facilities had played a significant role in community health in terms of providing access 
to curative and other services. 


They however complained that user fees and charges for extra investigations were 
unaffordable. 


Even then, hospitals and health centres still remained the first preference among the 
community members. 


They also complained of understaffing in terms of doctors and other relevant health 
staff. This was also followed by an out cry over the alleged poor attitude by some 
health workers, especially the so called Classified Daily Employee - CDEs, who included 
sweepers and other general workers. 


‘The issue of inadequate food for in-patients also came out prominently from the 
members of the community. 
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And the youths also complained that they were not fully involved in the decision 
_making structures of the Cord AID projects. 


From these findings, the participants noted that the issue of lack of youth involvement 
in decision-making in health institutions was a problem not unique to C.H.Is. They 
said this was even reflective in the political structures of the country, where mot of the 
people in decision-making positions were elderly people. 


Over the alleged poor attitude of CDEs, it was concluded that the lack of a deliberate 
policy to build the capacity of the CDEs at the point of recruitment made the situation 


grave. 


It was suggested that was a need to have these CDEs taken for workshops where they 
could be trained on basic patient handling skills, in line with the general ethics of 


medical practice. 


However, the major reason attributed to this alleged rough attitude of CDEs as well as 
some professional nurses was the environment under which they operated. These people 
were reportedly frustrated fellows, who lacked any motivation in their jobs. They end 
up being arrogant in responding to patients due to the pressure of work they have, yet 
without matching incentives. 


In some health centres, it was reported that nurses even do the job of doctors, but the 
pay they got at the end of the day did not tally with what they put in. This was linked 
to the issue of understaffing, with some reasons already alluded earlier on in the report. 


However, another reason attributed to understaffing in C.H.Is was the alleged 
government policy of doing away with the training of Enrolled Nurses, in preference to 
registered ones. It was reported better when churches were training their own enrolled 
nurses. But with this government policy, it has become impossible for them to continue 
staffing themselves. 


The failure also by the government to put nurses on the pay roll has compounded the 
problem of understaffing. 


4 Conclusion 


From the findings of this research and the analyses that were made by the Cord AID 
partners, it is evident that despite a number of positive strides having been made, much 
still remained to be done. 


The issues that were raised by the respondents in the research, and the partners clearly 
Showed that there was need for another touch in some points. 
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Recommendations 


Following their analyses, the participants came up with some recommendations 


that could help come up with a clearer vision. The following were some of the 
recommendations: 


a Cord AID may consider funding a meetin 
talk about the difficulties of C.H.Is to eas 
government and C.H.Is. 


b Cord AID should have consistency in their staffing in order to build rapport and a 
lasting relationship with their partners in Zambia. This could be achieved by having 
full-time employees in the same positions for at least three (3) years to promote 
progress and continuity. 

¢ Cord AID staff need to get down to the ground in order for them to have first hand 
experience of what actually goes on in the project areas in Zambia, and appreciate 
some of the difficulties they go through. 

d There is need to think of having different plpeople to be in charge of different 
programmes in Zambia, unlike the current situation where one person is in charge of 
all Cord AID sponsored projects like agriculture, health and others. 

e Cord AID needs to appreciate the strengths and weaknesses of the local environment. 
This will improve the relationship between Cord AID and the local partners. This 
may help reduce the misconceptions currently existing, thus reduce probable future 
misunderstandings. 

f Cord AID to continue lobbying for more funding for projects in countries like Zambia 
so as to think of providing free medical services to the vulnerable communities who 
cannot pay user fees. 

g The funding to C.H.Is needs not be attached with strings and conditions, allowing them to 
decide priority areas as stipulated in their budgets. 


g between the church and the DHMTs to 
e the tension and misunderstanding between 


ie ar i BM aiinae 
ey: 98) seen coe. h.J40 tan —— 


ay 


phy Teqypr th: ot gattiee: total ve tiean — 

nied ved bor ai bivo. 2% <tdemad ai ome ey Tait ght ls neg pees 
a Ps ah ‘ P p. a 7 ~ fj . 

a Pipow (o maces nae in Wr aerate, omit beatin rie nw eesti CPT te 


? ha =: i j « . 
: fd 

y nil me Cv 4 | : : 
ap £3 . ee The oes ie we “ry , eee, See gi - 8 ’ 


7 ; > " 5 — = ne 
| seenaneis ae Xe pike ESOP TA ee 
: r ° ‘ ; i. - 
7 me oi a 7” <_ 7 
oe pe a a a ae ee et 
| : ? aie o- : =e ire. 
- — mT TH -! ON a ORO ye. ee AO a S aa he 
> 
a m i ‘ o ie. CF a . oe = . ~ ae ae ee ve if as Ji a oe n 
cee a? ; hh. * oe ee arc ; gem 2 - > Aq Ps wr. ified i. as nee a : = es ' 
mer” 
ve or = ‘ = oo aliens : wee ; 7 
FA es F ee = Gon ied 5 pagers =A Ey weg ee : ~ 
Ss ; “ bs rec een 
, é es Ce. imi» 
» 0 z 
J 
+ 
— , _ 
=} : Li 
. , ¥ 
“ 
i i 


reer oe meee p18 
Appendix 10 


Responsibilities, Costs and Time Line of Research 
Process 


General 

The research process is described in detail in the Plan of Approach of the CFP evalua- 
tions 2003-2006. This document includes for instance the description of the role of the 
Coordination Group and the External Reference Group and the CFP quality criteria. 


Division of responsibilities 


a The Coordination Group consisting of policy officers and theme experts of the 
CFA’s was responsible for the implementation of this evaluation. ICCO chaired the 
Coordination Group. The members were: 


ICCO: Harry Derksen, Assistant Director Research and Development 
Dieneke de Groot, Policy Officer Evaluation 
Mariecke van der Glas, Policy Officer Evaluation 
Cordaid: Rens Rutten, Policy officer Health 
Plan Nederland: Lis Ostergaard, Evelien Kamminga (Sept. ’03 - February ’04) 
MBN-secretariat: Mark Rietveld (until 31/12/2003) 
Marian Wiersinga (until 3/11/2004) 
Mieke van Tilburg (from 01/04/05) 


b The Executing Agency was HLSP (former IHSD). The team leader was Cindy Carlson, 
co-evaluator was Janneke Roos. Researchers and writers for the different reports 
were: 


Country report Zambia: Kwibisia Liywalii 

Country report DRC: _=Bonani Bukaka, Abraham Mifundu, Bingo Kimpwene, 
Bakutuvwidi Makani, Mputu Dieri and Janneke Roos, 

Country report Nepal: Raghav Raj Regmi, Prabha Devi Shresta and Sugan 
Bajiracharya 

Synthesis report: Cindy Carlson and HLSP Institute 


c The External Reference Group assessed process and results of the evaluation. Their 
assessment of the synthesis report can be found annex 1. The members are: 


e Dr. P. (Paul) Engel (chair), Director, European Centre for Development Policy 
Management (ECDPM) 

e Prof. Dr. A.P. (Anita) Hardon, Professor of Care and Healthcare, Faculty of Social 
and Behavioural Sciences, University of Amsterdam and Dean, Amsterdam School 
for Social Science Research (ASSR) 

e Dr. R. (Rekha) Wazir, Co-director International Child Development Initiatives (ICDI) 

e Dr. A.G. (Geske) Dijkstra, Senior Lecturer Economics, Faculty of Social Sciences, 
Erasmus University of Rotterdam 

e Prof. Dr. Ir. G.E. (Georg) Frerks, Professor of Disaster Studies, Wageningen 
University and Research Centre, Professor of Conflict Prevention and Conflict 
Management, University of Utrecht 
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e Prof. Dr. A.H.J. (Bert) Helmsing, Professor of Local & Regional Development, 
Institute of Social Studies (ISS), Professor of Local and Regional Planning, 


University of Utrecht (until 01/01/2005) 


Selection criteria consultants (basis quality criteria mentioned in the tender letter) 


knowledge health sector in developing countries . 
application of rights based approach and social equity perspective 
civil society development / analysis of the institutional environment 


gender analysis 
knowledge of an experience with evaluations (e.g. management and research 


capacities) 


Financial data — 
Financial data on contribution of Cordaid, ICCO and Plan Nederland to the organisations 


evaluated in this study. See Table 3, p. 42 and Table 3, p. 183. 


Financed by: Cordaid, ICCO en Plan Nederland 
Total costs of this evaluation: € 174.179 

Time line 

First meeting Coordination Group: 14-08-2003 

Final version by HLSP: 08-10-2005 


More detailed time line: 

e Start evaluation mid 2003 (140803); basis for evaluation is a background paper of 
the 3 CFAs on health and this evaluation; in January 2004 the ERG commented on 
this background paper 

¢ March 2004: start tender procedure. Tender procedure hampered by draw back of an 
interesting candidate; selection had to start anew. 

¢ Early July 2004: HLSP (former IHSD) selected; provision made for language problems 
desk study by adding a Dutch speaking co-researcher. HLSP made a master ToR 
(based on their technical Proposal Based on the tender); 

° Meeting with CG and evaluators to discuss master TOR and country TOR on the 3th 
of September 2004 

* mid September 2004: the ERG commented on this and their and the CG’s comments 
were taken in the final version of the master ToR 

* Start evaluation field work second half of 2004 

¢ First draft report: January 2005; CG met with evaluators to comment on report 

° New draft in March 2005; this version was shared with ERG; comments given by 
Anita Hardon (ERG). 

° New draft September 2005; this version still was disappointing since the main 
comments of the CG /ERG were not addressed Satisfactorily and this version was 
more or less the same as the March version 

* October 7, 2005: meeting with evaluator and technical back-up of HLSP, which lead to 
October version of report (17-10-05) which was accepted by the CG and send to the ERG 


* November 8, 2005, meeting with ERG on final report of the evaluation; acceptance 
ERG. 
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